Skip to content

Courses
Tutorials
Practice
Contests

Beautiful Soup
Selenium
Scrapy
urllib
Request
open cv
Data analysis
Machine learning
NLP
Deep learning
Data Science
Interview question
ML math
ML Projects
ML interview
DL interview

Python | Extract URL from HTML using lxml

Extract CSS tag from a given HTML using Python

Last Updated : 24 Oct, 2020

Comments

Improve

Suggest changes

Like Article

Like

Report

Prerequisite: Implementing Web Scraping in Python with BeautifulSoup

In this article, we are going to see how to extract CSS from an HTML document or URL using python.

Module Needed:

bs4: Beautiful Soup(bs4) is a Python library for pulling data out of HTML and XML files. This module does not come built-in with Python. To install this type the below command in the terminal.

pip install bs4

requests: Requests allows you to send HTTP/1.1 requests extremely easily. This module also does not comes built-in with Python. To install this type the below command in the terminal.

pip install requests

Approach:

Import module
Create an HTML document and specify the CSS tag into the code
Pass the HTML document into the Beautifulsoup() function
Now traverse the tag with the select() method.

Implementation:

Python3

# import module 
from bs4 import BeautifulSoup 
  
# Html doc 
html_doc = """ 
<html> 
<head> 
<title>Geeks</title> 
</head> 
<body> 
<h2>paragraphs</h2> 
  
<p>Welcome geeks.</p> 
  
  
<p>Hello geeks.</p> 
  
<a class="example" href="www.geeksforgeeks.com" id="dsx_23">java</a> 
<a class="example" href="www.geeksforgeeks.com/python"  id="sdcsdsdf">python</a> 
</body> 
</html> 
"""
soup = BeautifulSoup(html_doc, "lxml") 
  
# traverse CSS from soup 
print("display by CSS class:") 
print(soup.select(".example")) 

Output:

display by CSS class:  [<a class="example" href="www.geeksforgeeks.com" id="dsx_23">java</a>,   <a class="example" href="www.geeksforgeeks.com/python" id="sdcsdsdf">python</a>]

Now let’s get the CSS tag with URL:

Python3

# import module 
from bs4 import BeautifulSoup 
import requests 
  
# link for extract html data 
# Making a GET request  
      
def getdata(url): 
    r=requests.get(url) 
    return r.text 
html_doc = getdata('https://www.geeksforgeeks.org/') 
soup = BeautifulSoup(html_doc,"lxml") 
  
# traverse CSS from soup 
  
print("\nTags by CSS class:") 
print(soup.select(".header-main__wrapper"))

Output:

Python | Extract URL from HTML using lxml

kumar_satyam

Improve

Article Tags :

Python
python-utility
Web-scraping

Practice Tags :

python

Similar Reads

Python | Extract URL from HTML using lxml

Link extraction is a very common task when dealing with the HTML parsing. For every general web crawler that's the most important function to perform. Out of all the Python libraries present out there, lxml is one of the best to work with. As explained in this article, lxml provides a number of help

Extract title from a webpage using Python

Prerequisite Implementing Web Scraping in Python with BeautifulSoup, Python Urllib Module, Tools for Web Scraping In this article, we are going to write python scripts to extract the title form the webpage from the given webpage URL. Method 1: bs4 Beautiful Soup(bs4) is a Python library for pulling

Extract JSON from HTML using BeautifulSoup in Python

In this article, we are going to extract JSON from HTML using BeautifulSoup in Python. Module neededbs4: Beautiful Soup(bs4) is a Python library for pulling data out of HTML and XML files. This module does not come built-in with Python. To install this type the below command in the terminal.pip inst

Find the title tags from a given html document using BeautifulSoup in Python

Let's see how to Find the title tags from a given html document using BeautifulSoup in python. so we can find the title tag from html document using BeautifulSoup find() method. The find function takes the name of the tag as string input and returns the first found match of the particular tag from t

Get tag name using Beautifulsoup in Python

Prerequisite: Beautifulsoup Installation Name property is provided by Beautiful Soup which is a web scraping framework for Python. Web scraping is the process of extracting data from the website using automated tools to make the process faster. Name object corresponds to the name of an XML or HTML t

Email Id Extractor Project from sites in Scrapy Python

Scrapy is open-source web-crawling framework written in Python used for web scraping, it can also be used to extract data for general-purpose. First all sub pages links are taken from the main page and then email id are scraped from these sub pages using regular expression.Â This article shows the e

How to click a href link from bootstrap tabs using Python?

When it comes to manipulation with web content, no one beats Python. A link can be clicked using Python and selenium library. Installation 1.1 Selenium Bindings in PythonSelenium Python bindings provide a convenient API to access Selenium Web Driver like Firefox,Chrome,etc. Pip install Selenium 1.2

Extract the HTML code of the given tag and its parent using BeautifulSoup

In this article, we will discuss how to extract the HTML code of the given tag and its parent using BeautifulSoup. Modules Needed First, we need to install all these modules on our computer. BeautifulSoup: Our primary module contains a method to access a webpage over HTTP.pip install bs4lxml: Helper

Parsel: How to Extract Text From HTML in Python

Parsel is a Python library used for extracting data from HTML and XML documents. It provides tools for parsing, navigating, and extracting information using CSS selectors and XPath expressions. Parsel is particularly useful for web scraping tasks where you need to programmatically extract specific d

How to Extract Weather Data from Google in Python?

In this article, we will see how to extract weather data from google. Google does not have its own weather API, it fetches data from weather.com and shows it when you search on Google. So, we will scrape the data from Google, and also we will see another method to fetch a schematic depiction of a lo

Corporate & Communications Address:

A-143, 7th Floor, Sovereign Corporate Tower, Sector- 136, Noida, Uttar Pradesh (201305)

Registered Address:

K 061, Tower K, Gulshan Vivante Apartment, Sector 137, Noida, Gautam Buddh Nagar, Uttar Pradesh, 201305

Advertise with us

Company
About Us
Legal
Privacy Policy
In Media
Contact Us
Advertise with us
GFG Corporate Solution
Placement Training Program

Languages
Python
Java
C++
PHP
GoLang
SQL
R Language
Android Tutorial
Tutorials Archive

DSA
Data Structures
Algorithms
DSA for Beginners
Basic DSA Problems
DSA Roadmap
Top 100 DSA Interview Problems
DSA Roadmap by Sandeep Jain
All Cheat Sheets

Data Science & ML
Data Science With Python
Data Science For Beginner
Machine Learning
ML Maths
Data Visualisation
Pandas
NumPy
NLP
Deep Learning

Web Technologies
HTML
CSS
JavaScript
TypeScript
ReactJS
NextJS
Bootstrap
Web Design

Python Tutorial
Python Programming Examples
Python Projects
Python Tkinter
Python Web Scraping
OpenCV Tutorial
Python Interview Question
Django

Computer Science
Operating Systems
Computer Network
Database Management System
Software Engineering
Digital Logic Design
Engineering Maths
Software Development
Software Testing

DevOps
Git
Linux
AWS
Docker
Kubernetes
Azure
GCP
DevOps Roadmap

System Design
High Level Design
Low Level Design
UML Diagrams
Interview Guide
Design Patterns
OOAD
System Design Bootcamp
Interview Questions

Inteview Preparation
Competitive Programming
Top DS or Algo for CP
Company-Wise Recruitment Process
Company-Wise Preparation
Aptitude Preparation
Puzzles

School Subjects
Mathematics
Physics
Chemistry
Biology
Social Science
English Grammar
Commerce
World GK

GeeksforGeeks Videos
DSA
Python
Java
C++
Web Development
Data Science
CS Subjects

@GeeksforGeeks, Sanchhaya Education Private Limited, All rights reserved

We use cookies to ensure you have the best browsing experience on our website. By using our site, you acknowledge that you have read and understood our Cookie Policy & Privacy Policy

Improvement

Suggest Changes

Help us improve. Share your suggestions to enhance the article. Contribute your expertise and make a difference in the GeeksforGeeks portal.

geeksforgeeks-suggest-icon

Create Improvement

Enhance the article with your expertise. Contribute to the GeeksforGeeks community and help create better learning resources for all.

geeksforgeeks-improvement-icon

Suggest Changes

min 4 words, max Words Limit:1000

Thank You!

Your suggestions are valuable to us.

What kind of Experience do you want to share?

Interview Experiences

Admission Experiences

Career Journeys

Work Experiences

Campus Experiences

Competitive Exam Experiences