How to Get Only Certain Columns in Pandas

Pandas is a powerful Python library used for data manipulation and analysis. It often deals with large datasets. To work with such data effectively, isolation of the relevant information is crucial. This is where selecting specific columns comes into play. This guide will explore how to get only certain columns in pandas according to our requirements. To do this we will discuss different methods along with many associated things.

Understanding Pandas DataFrames

Before jumping into column selection, let’s briefly understand what a Pandas DataFrame is. Imagine a DataFrame as a spreadsheet-like structure with rows and columns. Each column represents a specific variable, and each row contains observations for that variable.

How to Get Only Certain Columns in Pandas? Step-by-Step guide.

Method 1: Selecting Columns Using Square Brackets

This is the most straightforward way to select columns. You can pass a column name or a list of column names inside square brackets.

import pandas as pd

# Sample DataFrame

data = {‘Name’: [‘Alice’, ‘Bob’, ‘Charlie’],

        ‘Age’: [25, 30, 28],

        ‘City’: [‘New York’, ‘Los Angeles’, ‘Chicago’]}

df = pd.DataFrame(data)

# Selecting a single column

selected_column = df[‘Age’]

print(selected_column)

# Selecting multiple columns

selected_columns = df[[‘Name’, ‘City’]]

print(selected_columns)

Use code with caution.

Method 2: Using the loc Attribute

The loc attribute is versatile for selecting data based on labels. To select columns, you can use a colon (:) to select all rows and specify the desired columns after the comma.

# Selecting specific columns using loc

selected_columns = df.loc[:, [‘Name’, ‘Age’]]

print(selected_columns)

Use code with caution.

Method 3: Using the iloc Attribute

The iloc attribute is used for integer-based indexing. On the other hand for selecting rows and columns by position, you can use iloc attribute to select columns by their integer index.

# Selecting columns by integer index using iloc

selected_columns = df.iloc[:, [0, 2]]  # Select columns at index 0 and 2

print(selected_columns)

Method 4: Filtering Columns Based on Conditions

You can filter columns using Boolean indexing or regular expressions.

# Filtering columns based on column names

selected_columns = df.filter(regex=’A.*’)  # Select columns starting with ‘A’

print(selected_columns)

# Filtering columns based on a list of column names

selected_columns = df.filter(items=[‘Name’, ‘City’])

print(selected_columns)

Additional Tips

• Column Order: The order of columns in the selected DataFrame will be the same as the order in the original DataFrame unless specified otherwise.
• Performance: For large DataFrames, using iloc might be slightly faster than loc.
• Copy vs. View:  Be sure to know the difference between creating a copy and a view of the original DataFrame. Use copy() if you want to modify the selected data without affecting the original DataFrame.

You may also like to read How to Select a Column in Pandas: Techniques and Tips

How do I extract certain columns in pandas?

To extract certain columns from a DataFrame, you can use double brackets with the column names you want to select. For example:

selected_columns = df[[‘column1’, ‘column2’]]

How do I print only certain columns in pandas?

You can print specific columns using the same method as extracting them. For example:

print(df[[‘column1’, ‘column2’]])

How do I show only a few columns in pandas?

To display only a few columns, again, you can use double brackets:

few_columns = df[[‘column1’, ‘column2’]]

print(few_columns)

How to get only column values in pandas?

If you want to get the values of a specific column, you can use the single bracket notation or the dot notation if the column name doesn’t contain spaces. To get the values as a numpy array, you can use .values or .to_numpy():

column_values = df[‘column1’].values  # or

column_values = df[‘column1’].to_numpy()

How do I exclude some columns in Pandas?

To exclude certain columns, you can use the drop method:

df_excluded = df.drop([‘column1’, ‘column2’], axis=1)

How to select columns in Pandas with condition?

To select columns based on a condition (e.g., data type), you can use boolean indexing or DataFrame methods. For example, to select columns with numeric data types:

numeric_columns = df.select_dtypes(include=’number’)

How to extract specific columns from CSV in Python?

You can read specific columns from a CSV by using the usecols parameter in pd.read_csv:

df = pd.read_csv(‘file.csv’, usecols=[‘column1’, ‘column2’])

How do I get a list of columns in a DataFrame?

To get a list of column names in a DataFrame, you can use the columns attribute:

column_list = df.columns.tolist()

Benefits of/Reasons for Selecting Only Certain Columns in Pandas

Selecting specific columns from a Pandas DataFrame is a basic operation for several reasons:

Memory Efficiency

Large Datasets:

When dealing with massive datasets, loading all columns into memory can be computationally expensive and time-consuming.

Unnecessary Data:

Many datasets contain some columns that are irrelevant to the current analysis. By selecting only the required columns, you reduce memory usage significantly.

Performance Improvement

Faster Operations:

Working with a smaller subset of data often leads to faster computations, especially when performing calculations or aggregations.

Optimized Algorithms:

Some algorithms and libraries are optimized for smaller datasets. Selecting relevant columns can enhance their performance.

Data Privacy

• Sensitive Information: In many cases, datasets contain sensitive information that should not be exposed. Selecting the necessary columns, you can protect sensitive data 

Focus on Specific Analysis

Targeted Insights:

Often, you’re interested in a particular aspect of the data. Selecting relevant columns helps you focus on the specific analysis without distractions.

Feature Engineering:

For machine learning models, selecting the most informative features is crucial for model performance.

Data Preparation

Data Cleaning:

You might need to clean or preprocess certain columns before further analysis. Selecting these columns allows you to work on them independently.

Data Transformation:

Creating new features or transforming existing ones often involves selecting specific columns as input.

Visualization

• Clarity: Visualizing all columns in a dataset can be overwhelming. Selecting a few key columns improves the clarity and interpretability of visualizations.
• Storytelling: By focusing on specific columns, you can create compelling visualizations that tell a clear story about the data.

Exporting Data

Smaller File Size: Exporting only the necessary columns creates smaller files, reducing storage space and transfer time.

Specific Requirements: Some systems or applications might have limitations on the number of columns or data size.

Joining with Other Dataframes

• Common Columns: When merging or joining DataFrames, you often need to align them based on common columns. Selecting these columns beforehand facilitates the process.

Conclusion

Mastering column selection in Pandas is a fundamental skill. It enhances your capabilities of data manipulation. By understanding how to get only certain columns in pandas in different methods and techniques, you can efficiently work with your datasets. Focusing on this you can manage data more conveniently. 

Always practice with your datasets to solidify your understanding.

FAQs on How to Get Only Certain Columns in Pandas