Polars sort a dataframe

Polars DataFrame sort method has the following typing:

DataFrame.sort(
    by: IntoExpr | Iterable[IntoExpr],
    descending: bool | Sequence[bool] = False,
    nulls_last: bool = False,
) → DataFrame

The descending and null_last arguments have a default False behaviour.

Let’s deep into it and start by create a simple Polars dataframe and check the sorting basics. You can call it directly over an string representing a column name, or sort it by using a pl.col() structure.

import polars as pl

df = pl.DataFrame(
    {
        "Company": ["Tesla", "Tesla_old", "Apple", "Microsoft"],
        "Market_Cap": [0.798, None, 2.78, 2.44],
        "Diluted_EPS": [0.79, None, 1.26, 2.69],
    }
)

df.sort("Market_Cap")

df.sort("Market_Cap").frame_equal(df.sort(pl.col("Market_Cap"))) # Comparte to check it is equal

## True

Check that as expected, the default behaviour is to place nulls at the beginning and sort values in ascending way.

df.sort(
    ["Market_Cap", "Diluted_EPS"],
    descending=True,
    nulls_last=True # not needed when descending is True
)

df.sort(
    pl.col("Market_Cap") / pl.col("Diluted_EPS"), 
    descending=True, 
)

df.select(pl.col(["Company","Diluted_EPS"]).sort())

# You can do smth similar by packing them inside an Struct
# df.select(pl.struct(["Company", "Diluted_EPS"]).sort())

Polars sorted flag

In Polars, the use of a “sorted” flag comes in handy when you want to explicitly indicate that a column has been sorted, especially when it pertains to data generated, f.e over a range of date. This flag is automatically applied when you’ve used the sort() expression. This flag serves as a performance-enhancing tool, optimizing subsequent operations and enabling certain functions to work more efficiently when they require the data to be in a sorted state before their execution.

Let’s take a look at an illustrative example:

df["Diluted_EPS"].is_sorted()

## False

df_not_sorted_but_flagged_as_sorted = df.with_columns(pl.col("Diluted_EPS").set_sorted())
df_not_sorted_but_flagged_as_sorted["Diluted_EPS"].is_sorted()

## True

Remember te reasignation as Polars does not work with inplace operations.

df_sorted = df.sort("Diluted_EPS", descending=True)
print(df_sorted["Diluted_EPS"].is_sorted())

## False

What’s happening here?? We can access to Polars column flags by doing:

df_sorted["Diluted_EPS"].flags

## {'SORTED_ASC': False, 'SORTED_DESC': True}

The proper way to verify this is to:

print(df_sorted["Diluted_EPS"].is_sorted(descending=True))

## True

any(df_sorted["Diluted_EPS"].flags) # Or directly if you don't know the order

## True

Stay updated on Polars and Python tips

Hopefully, this post has helped you become familiar with Polars sort usage and allowed you to enjoy a showcase of some of its features.

If you want to stay updated…