CodeRigor

LESSON

Aggregate at an explicit grain

Distinguish row count, known-value count and all-missing totals, including unknown key groups and overflow.

State the output grain before aggregating: one row per customer key in this exercise. Then distinguish how many observations exist from how many values are known. Group size includes rows whose amount is missing; count includes only known amounts.

A group with two missing amounts is not a group totaling zero. Use min_count=1 to preserve that distinction. Also decide what happens to missing group keys and arithmetic overflow; defaults should not silently discard or wrap evidence.

Aggregate at an explicit grain
start / 3
Distinguish row count, known-value count and all-missing totals, including unknown key groups and overflow.
Aggregate at an explicit grain
Distinguish row count, known-value count and all-missing totals, including unknown key groups and overflow.

Cast cents to object/Python integers for summation before converting the reviewed result back to nullable Int64. This avoids a signed64-bit accumulator silently wrapping. A group whose exact total is outside the chosen output range is rejected explicitly.

dropna=False retains an unknown-key group in this analytical exercise. The later merge contract rejects unknown keys when establishing relationships. Grouping policy and join policy serve different purposes and should not be conflated.

Python
import pandas as pd
frame=pd.DataFrame({"key":["zero","missing","missing"],"cents":pd.array([0,None,None],dtype="Int64")})
groups=frame.groupby("key",dropna=False)
print(groups.size().to_dict())
print(groups["cents"].count().to_dict())
print(groups["cents"].sum(min_count=1).isna().to_dict())

The output exposes rows, known_cents and total_cents separately. A zero total with one known observation differs from an unknown total with two missing observations. Keep both the count semantics and nullable result dtype visible in the report.

Aggregating at the wrong grain or dropping unknown keys can change later reconciliation. Test signed values, empty input, all-missing groups, zero groups and an overflow counterexample instead of only a convenient positive dataset.

Keep the interpretation visible
start / 3
Column representation, accepted business policy and independent report checks are separate decisions.
Keep the interpretation visible
Column representation, accepted business policy and independent report checks are separate decisions.
Python
import pandas as pd
values=pd.Series(pd.array([2**63-1,1],dtype="Int64"))
exact=values.astype(object).sum()
print(exact)
print("fits signed64",-(2**63)<=exact<=2**63-1)

Implement the rollup with count and missing semantics visible. Reject overflow in this bounded schema. Next enrich rows through a validated relationship.

Watch & read

Free lectures, talks and readings that cover this skill from another angle. All resources →