Polars: Choosing Columns with Square Brackets

This approach comes with certain limitations and is best suited for interactive and exploratory coding.

Let’s explore some examples after creating the base DataFrame (note it’s not Lazy for this time, which you usually will use while working with high amounts of data):

import polars as pl
import random 

# Create a Polars DataFrame with base columns
df = pl.DataFrame({
    'name': ['Alice', 'Bob', 'Charlie'], 
    'offensive_skill': [5, 30, 85], 
    'defensive_skill': [92, 30, 10]
    },
    schema={
        "name":pl.Utf8,
        "offensive_skill":pl.Int32,
        "defensive_skill":pl.Int32
    }
)

We can select a column using square brackets with a string inside []. Note that the output is a Polars Series.

df["name"].head(3)

We can choose a column with a list of strings inside []. We should expect now a Polars DataFrame as an output:

df[["name","defensive_skill"]].head(3)

The major caveats about square brackets column selection is that it can only be used in eager mode. So let’s start and deepen into the native Polars select expression.

Polars Select

One of the key benefits of using Polars select function is that it can be employed in lazy mode, allowing for optimization and parallel execution by Polars.

It’s essential to note that this method consistently yields a Polars DataFrame.

To effortlessly choose specific columns, simply pass their string names as a list to the Polars select function.

(
    df
    .select(
        ["name", "defensive_skill"]
    )
    .head(3)
)

In the next paragraphs we will discover that this is just a simple way to do it, as you usually would use pl.col() inside select() function.

df.select(
    "^.*skill$"
)

(
    df
    .select(
        pl.col("defensive_skill").mean().alias("defensive_skill_mean"),
        pl.col("defensive_skill").std().alias("defensive_skill_std"),
        pl.col("offensive_skill").mean().suffix("_mean"), # shorter to add a suffix to a column name
        pl.col("offensive_skill").std().suffix("_std")
    )
    .head(3)
)

df.select(
    pl.all()
)

df.select(
    pl.exclude("defensive_skill") 
    #pl.exclude(["name", "defensive_skill"]) # pass a list for select several columns
)

df.select(
    pl.col(pl.Utf8)
    #pl.col([pl.Utf8, pl.Int32]) # several types as a list
    #pl.col(pl.NUMERIC_TYPES) # all numeric types
)

Polars selectors API

Enhance your data analysis efficiency with Selectors! Selectors provide a convenient way to select columns from DataFrame or LazyFrame objects based on their name, data type, or other attributes. They streamline and extend the functionality offered by the col() expression, while also enabling the easy application of expressions to the selected columns. Say goodbye to tedious column selection and hello to the simplicity of Selectors!

import polars.selectors as cs

One could use this API to easily select columns by type as the example above, but in a simpler / more readable way:

df.select(
    cs.string()
)

For datetime and timezone selection and manipulation with Polars, please find our post about selecting datetimes and timezones with Polar selectors

df.select(
    cs.datetime()
)

A summary of all selector functions:

All of them could be find in the Polars selectors documentation