CodeRigor

LESSON

Select, sort and derive columns without losing alignment

Use nullable masks, stable ordering and exact integer-derived currency text while preserving the source index.

A mask selects rows; sorting changes their order; a derived column must still line up with those selected observations. Pandas aligns Series by index labels, which is useful when intentional and surprising when a fresh default-index Series is assigned to a frame with old labels.

The exercise selects known cents above a threshold, sorts with a stable tie policy and formats currency using integer arithmetic. It preserves the source index as evidence of which observations remain. It does not reset labels and pretend the row positions are the original identities.

Select, sort and derive columns without losing alignment
start / 3
Use nullable masks, stable ordering and exact integer-derived currency text while preserving the source index.
Select, sort and derive columns without losing alignment
Use nullable masks, stable ordering and exact integer-derived currency text while preserving the source index.

Nullable comparisons can produce pd.NA. Combine the nonmissing check with the comparison and fill an unknown mask value with False. Do not rely on Python truthiness to select data with nullable scalars.

Sort descending cents, then ascending sale_id. Use a stable sort policy and preserve original labels. Assign derived text using a positional Pandas array so index alignment cannot put an otherwise correct amount on the wrong row.

Python
import pandas as pd
frame=pd.DataFrame({"id":["b","a","c"],"cents":pd.array([29,29,None],dtype="Int64")},index=[10,20,30])
mask=frame["cents"].notna() & frame["cents"].ge(0).fillna(False)
result=frame.loc[mask].sort_values(["cents","id"],ascending=[False,True],kind="stable")
print(result["id"].tolist(),result.index.tolist())

The derived label uses sign, absolute integer cents, whole units and a two-digit remainder. It preserves zero and refunds without binary float conversion. The result is a new frame, while the input retains its columns, values and index.

Read the index as labels when using loc. Use iloc for positional inspection. Reset an index only when that is an intentional new convention; do not confuse resetting labels with repairing a broken alignment.

Keep the interpretation visible
start / 3
Column representation, accepted business policy and independent report checks are separate decisions.
Keep the interpretation visible
Column representation, accepted business policy and independent report checks are separate decisions.
Python
for cents in (0,29,-125):
    value=abs(cents)
    print(("-" if cents<0 else "")+f"${value//100}.{value%100:02d}")

Implement the mask, tie ordering and integer formatting. Preserve source labels and verify both a refund and an empty result. Next retain why an amount is absent.

Actual Pandas 2.3.3 · Independent records comparison

Follow the same observations through each operation.

These tables were computed from the supplied fixture. Choose a step to inspect actual values and row counts. Full report parity includes accepted rows, rejection reasons, unmatched source positions and every summary field.

11source records
5accepted rows
6rejected rows
1unmatched accepted
Raw CSV · 11 rows

Eleven raw records retain text IDs, literal NA and empty amounts before business validation.

Raw CSV · actual fixture values; null is labelled missing and empty text stays distinct
sale_idcustomer_iddateamount
s0010012025-01-01$0.29
s0020022025-01-02$0
s003missing2025-01-03-$1.25
s0010012025-01-01$0.29
s0010012025-01-01$0.30
s004"" · empty text2025-01-04$4
s005NA2025-01-05$2
s0060012025-02-30$1
s0070012025-01-07$1,234.50
s0080012025-01-08bad
s0090012025-01-09"" · empty text
Accepted lesson-frame dtypessale_id · stringcustomer_id · stringdate · stringcents · Int64
Download the data, independent baseline, Pandas implementation and checks →

Watch & read

Free lectures, talks and readings that cover this skill from another angle. All resources →