How to get data from LinkedIn using Python
Last Updated : 30 Jan, 2023
Linkedin is a professional tool that helps connect people of certain industries together, and jobseekers with recruiters. Overall, it is the need of an hour. Do you have any such requirement in which need to extract data from various LinkedIn profiles? If yes, then you must definitely check this article.
Stepwise Implementation:
Step 1: First of all, import the library's selenium and time. And most importantly we would require a web driver to access the login page and all.
Python3 from time import sleep from selenium import webdriver from selenium.webdriver.chrome.service import Service from selenium.webdriver.common.by import By from webdriver_manager.chrome import ChromeDriverManager
Step 2: Now, declare the webdriver and make it headless, i.e., run the web driver in the background. Here we will be using the ChromeOptions feature from the web driver by making it headless().
Python3 options = webdriver.ChromeOptions() options.add_argument("headless") exe_path = ChromeDriverManager().install() service = Service(exe_path) driver = webdriver.Chrome(service=service, options=options) Step 3: Then, open the website from which you want to obtain table data and make Python sleep for some time so that the page gets loaded quickly. Sleep is used to stop the program for that much time so, that the website that has been loaded gets loaded completely.
Python3 driver.get("https://www.linkedin.com/login") sleep(5) Now let's define a LinkedIn user id and LinkedIn password for that mail id.
Python3 # login credentials linkedin_username = "#Linkedin Mail I'd" linkedin_password = "#Linkedin Password"
Step 4: Further, automate entering your Linkedin account email and password and make Python sleep for a few seconds. This step is very crucial because if the login failed anyhow then none of the code after this will work and the desired output will not be shown.
Python3 driver.find_element_by_xpath( "/html/body/div/main/div[2]/div[1]/form/div[1]/input").send_keys(linkedin_username) driver.find_element_by_xpath( "/html/body/div/main/div[2]/div[1]/form/div[2]/input").send_keys(linkedin_password) sleep( # Number of seconds) driver.find_element_by_xpath( "/html/body/div/main/div[2]/div[1]/form/div[3]/button").click()
Step 5: Now, create a list containing the URL of the profiles from where you want to get data.
Python3 profiles = ['#Linkedin URL-1', '#Linkedin URL-2', '#Linkedin URL-3']
Step 6: Create a loop to open all the URLs and extract the information from them. For this purpose, we will use the driver function which will automate the search work by performing the operations iteratively without any external interference.
Python3 for i in profiles: driver.get(i) sleep(5)
Step 6.1: Next, obtain the title and description from all the profiles.
Python3 title = driver.find_element_by_xpath( "//h1[@class='text-heading-xlarge inline t-24 v-align-middle break-words']").text print(title)
Step 6.2: Now, obtain the description of each person from the URLs.
Python3 description = driver.find_element_by_xpath( "//div[@class='text-body-medium break-words']").text print(description) sleep(4)
Step 7: Finally, close the browser.
Python3 Example:
Python3 from time import sleep from selenium import webdriver from selenium.webdriver.chrome.service import Service from selenium.webdriver.common.by import By from webdriver_manager.chrome import ChromeDriverManager options = webdriver.ChromeOptions() options.add_argument("headless") exe_path = ChromeDriverManager().install() service = Service(exe_path) driver = webdriver.Chrome(service=service, options=options) driver.get("https://www.linkedin.com/login") sleep(6) linkedin_username = "#Linkedin Mail I'd" linkedin_password = "#Linkedin Password" driver.find_element(By.XPATH, "/html/body/div/main/div[2]/div[1]/form/div[\ 1]/input").send_keys(linkedin_username) driver.find_element(By.XPATH, "/html/body/div/main/div[2]/div[1]/form/div[\ 2]/input").send_keys(linkedin_password) sleep(3) driver.find_element(By.XPATH, "/html/body/div/main/div[2]/div[1]/form/div[\ 3]/button").click() profiles = ['https://www.linkedin.com/in/vinayak-rai-a9b231193/', 'https://www.linkedin.com/in/dishajindgar/', 'https://www.linkedin.com/in/ishita-rai-28jgj/'] for i in profiles: driver.get(i) sleep(5) title = driver.find_element(By.XPATH, "//h1[@class='text-heading-xlarge inline t-24 v-align-middle break-words']").text print(title) description = driver.find_element(By.XPATH, "//div[@class='text-body-medium break-words']").text print(description) sleep(4) driver.close() Output:
Similar Reads
Automatically Get Top 10 Jobs from LinkedIn Using Python
Here we are going to use Clicknium to scrape LinkedIn top 10 jobs. First, we will login to LinkedIn to search the jobs according to the job keyword(the title, the skill, or the company) and the location, and then get the top 10 jobs in the search results. For each job, we will get the job informatio
12 min read
How to Get Data from API in Python Flask
In modern web development, APIs (Application Programming Interfaces) play a crucial role in enabling the interaction between different software systems. Flask, a lightweight WSGI web application framework in Python, provides a simple and flexible way to create APIs. In this article, we'll explore ho
2 min read
How to get the Daily News using Python
In this article, we are going to see how to get daily news using Python. Here we will use Beautiful Soup and the request module to scrape the data. Modules neededbs4: Beautiful Soup(bs4) is a Python library for pulling data out of HTML and XML files. This module does not come built-in with Python. T
3 min read
Automate linkedin connections using Python
Automating LinkedIn connections using Python involves creating a script that navigates LinkedIn, finds users based on specific criteria (e.g., job title, company, or location), and sends personalized connection requests. In this article, we will walk you through the process, using Selenium for web a
5 min read
How to Make API Call Using Python
APIs (Application Programming Interfaces) are an essential part of modern software development, allowing different applications to communicate and share data. Python provides a popular library i.e. requests library that simplifies the process of calling API in Python. In this article, we will see ho
3 min read
How to read Emails from Gmail using Gmail API in Python ?
In this article, we will see how to read Emails from your Gmail using Gmail API in Python. Gmail API is a RESTful API that allows users to interact with your Gmail account and use its features with a Python script. So, let's go ahead and write a simple Python script to read emails. RequirementsPytho
6 min read
How to Extract Weather Data from Google in Python?
In this article, we will see how to extract weather data from google. Google does not have its own weather API, it fetches data from weather.com and shows it when you search on Google. So, we will scrape the data from Google, and also we will see another method to fetch a schematic depiction of a lo
4 min read
How to extract youtube data in Python?
Prerequisites: Beautifulsoup YouTube statistics of a YouTube channel can be used for analysis and it can also be extracted using python code. A lot of data like viewCount, subscriberCount, and videoCount can be retrieved. This article discusses 2 ways in which this can be done. Method 1: Using YouTu
3 min read
Extract CSS tag from a given HTML using Python
Prerequisite: Implementing Web Scraping in Python with BeautifulSoup In this article, we are going to see how to extract CSS from an HTML document or URL using python. Module Needed: bs4: Beautiful Soup(bs4) is a Python library for pulling data out of HTML and XML files. This module does not come bu
2 min read
How to Add Github to Linkedin?
Adding a GitHub profile to LinkedIn is a great way to showcase our coding projects, contributions, and technical skills. This can enhance our LinkedIn profile by providing potential employers and professional connections with direct access to our work. There are multiple ways to add your GitHub to L
2 min read