Reading specific columns of a CSV file using Pandas

Last Updated : 21 Nov, 2024

When working with large datasets stored in CSV (Comma-Separated Values) files, it’s often unnecessary to load the entire dataset into memory. Instead, you can selectively read specific columns using Pandas in Python.

Read Specific Columns From CSV File

Let us see how to read specific columns of a CSV file using Pandas. This can be done with the help of the pandas.read_csv() method. We will pass the first parameter as the CSV file and the second parameter as the list of specific columns in keyword usecols. It will return the data of the CSV file of specific columns.

The usecols parameter in the read_csv() function filters the columns to be loaded into the DataFrame. This is particularly useful when:

The dataset contains hundreds or thousands of columns.
You are only interested in analyzing a subset of the data.

Below are some examples by which we can read specific columns of a CSV file using Pandas.

Read Entire Columns of a CSV File

In this example, the Pandas library is imported, and the code reads the entire content of the “student_scores2.csv” file into a DataFrame ‘df’ using Pandas. The printed output displays the entire dataset for further examination.

Python

import pandas as pd # read specific columns of csv file using Pandas df = pd.read_csv("student_scores2.csv") print(df)

Output:

Screenshot-2023-11-24-132727

Read Specific Columns of a CSV File Using usecols

In this example, the Pandas library is imported, and the code uses it to read only the ‘IQ’ and ‘Scores’ columns from the “student_scores2.csv” file, storing the result in the DataFrame ‘df’. The printed output displays the selected columns for analysis.

Python

import pandas as pd  # read specific columns of csv file using Pandas df = pd.read_csv("student_scores2.csv", usecols=['IQ', 'Scores']) print(df)

Output:

With another example, the code reads the ‘Survived’ and ‘Pclass’ columns from the “titanic.csv” file using Pandas. The resulting DataFrame ‘df’ displays the selected columns for analysis.

Python

import pandas as pd   # read specific columns of csv file using Pandas df = pd.read_csv("titanic.csv", usecols = ['Survived','Pclass']) print(df)

Output:

Selecting Specific Columns by Index

If you don’t know the column names or prefer working with indices, you can pass a list of integers representing column positions:

Python

# Read only the thid and last column (indices 2 and 3) df = pd.read_csv('student_scores2.csv', usecols=[2, 3]) df

Output:

	Scores	Pass
0	18	0
1	45	1
2	25	0
3	72	1
4	30	0
5	20	0
6	88	1
..
..

Using Lambda for Dynamic Selection

You can also use functions or regular expressions to dynamically select columns. For example:

Python

# Dynamically select columns containing "Name" or "Salary" df = pd.read_csv('/content/student_scores2.csv', usecols=lambda col: 'IQ' in col or 'Pass' in col) df

Output:

How to skip rows while reading csv file using Pandas?

vipinyadav15799

Improve

Article Tags :

To read specific columns from an Excel file in Pandas, you have the flexibility to use either column indices or letters. This is achieved by setting the usecols argument, which can take a comma-separated string or a list containing column identifying letters or indices. In this article, we will lear

5 min read

How to skip rows while reading csv file using Pandas?

Python is a good language for doing data analysis because of the amazing ecosystem of data-centric python packages. Pandas package is one of them and makes importing and analyzing data so much easier. Here, we will discuss how to skip rows while reading csv file. We will use read_csv() method of Pan

3 min read

Replacing column value of a CSV file in Python

Let us see how we can replace the column value of a CSV file in Python. CSV file is nothing but a comma-delimited file. Method 1: Using Native Python wayÂ Using replace() method, we can replace easily a text into another text. Â In the below code, let us have an input CSV file as "csvfile.csv" and be

2 min read

Convert Text File to CSV using Python Pandas

Converting Text File to CSV using Python Pandas refers to the process of transforming a plain text file (often with data separated by spaces, tabs, or other delimiters) into a structured CSV (Comma Separated Values) file using the Python Pandas library. In this article we will walk you through multi

2 min read

Export Specific Columns in DataFrame to CSV File

A data frame is a tabular data structure similar to a spreadsheet. It has rows and columns. CSV (Comma-Separated Values) is a plain text file format used for storing tabular data. Each line in a CSV file represents a row of data, and the values within a line are separated by commas. This article exp

3 min read

Rename column name with an index number of the CSV file in Pandas

In this blog post, we will learn how to rename the column name with an index number of the CSV file in Pandas. Renaming Column Name with an Index NumberPandas, an advanced data manipulation package in Python, includes several methods for working with structured data such as CSV files. You might wish

6 min read

Select Columns with Specific Data Types in Pandas Dataframe

In this article, we will see how to select columns with specific data types from a dataframe. This operation can be performed using the DataFrame.select_dtypes() method in pandas module. Syntax: DataFrame.select_dtypes(include=None, exclude=None)Parameters :Â include, exclude : A selection of dtypes

2 min read

Get a List of a Specific Column of a Pandas DataFrame

In data analysis, extracting specific columns from a DataFrame and converting them into Python lists is a common requirement. Pandas provides multiple ways to achieve this efficiently. This article explores various methods to extract a specific column from a Pandas DataFrame and convert it into a li

3 min read

How to Select Single Column of a Pandas Dataframe

In Pandas, a DataFrame is like a table with rows and columns. Sometimes, we need to extract a single column to analyze or modify specific data. This helps in tasks like filtering, calculations or visualizations. When we select a column, it becomes a Pandas Series, a one-dimensional data structure th

2 min read

How to Convert Pandas Columns to String

Converting columns to strings allows easier manipulation when performing string operations such as pattern matching, formatting or concatenation. Pandas provides multiple ways to achieve this conversion and choosing the best method can depend on factors like the size of your dataset and the specific

3 min read

Reading specific columns of a CSV file using Pandas

Read Specific Columns From CSV File

Read Entire Columns of a CSV File

Read Specific Columns of a CSV File Using usecols

Selecting Specific Columns by Index

Using Lambda for Dynamic Selection

Similar Reads