A schema is more than a list of field names. It describes allowed runtime types,
required values, permitted spellings, normalization and failure behavior. If you
write int(value) and accept everything it accepts, the constructor has quietly
become your business policy. int(True) yields 1; int(2.9) truncates to 2.
Neither is acceptable when a feed promises a nonnegative integer visit count.
Start from the existing Python record lessons: annotations help readers and
tools, but they do not validate incoming dictionaries. The build accepts exactly
two keys, id and visits. An unexpected field is reported rather than projected
away. This catches a misspelled or newly introduced column before an analysis
quietly ignores it. Rejecting unknown fields is one useful policy, not a universal
requirement for every data source.
Match the allowed set before converting
import re
for value in [True, 2, ' 002 ', '+2', '2']:
integer = type(value) is int and value >= 0
digit_text = type(value) is str and re.fullmatch(r'[0-9]+', value.strip()) is not None
print(repr(value), bool(integer or digit_text))The count policy accepts an exact integer or stripped ASCII digit text. It
rejects signs, decimal notation, Unicode digits and booleans. The exact-type
check is deliberate because bool is an int subclass. A different schema may
permit these conversions, but it should say so and test them explicitly.
The identifier policy differs from the count policy. Strip boundary whitespace
from " 001 " to get "001", while preserving leading zeros, case and internal
spaces. An identifier is a label, so converting it to an integer can merge two
entities that the source meant to keep distinct. Normalization is information
loss; justify it field by field.
Keep a ledger instead of a broad exception blanket
coerce_visits returns accepted records and rejection records. Every rejection
contains the original 1-based row position and a short reason. If a row has an
extra field, an empty id and an invalid count, it receives schema: the contract
chooses the first failure in a fixed order. This avoids unstable diagnostics
and teaches a simple function; a comprehensive multi-error report would need a
different output contract.
Catch the expected conversion error at the conversion boundary. Avoid wrapping
the entire function in except Exception, which can turn a programming defect
into an ordinary rejected input. Do not mutate source dictionaries while
normalizing them. Fresh accepted dictionaries let callers compare the original
spelling with the result.
Check the reconciliation equation after any batch: input rows equal accepted rows plus rejected rows. An empty accepted list alone does not tell you whether the source was empty or every row failed. The separate ledger resolves that ambiguity. In practice, try a zero count, a boolean, a Unicode digit, an extra column and a missing id before filling the function.