CodeRigor

FREE LESSON

Idempotency keys & effect deduplication

Apply an account effect once per stable message key and reject conflicting key reuse without partial state.

A worker can commit an effect and lose its acknowledgement. On redelivery, neither the broker nor a delivery token alone tells the application whether that effect already happened. Idempotent effect handling gives repeated presentations of one logical operation the same result without another balance change. It does not require the transport to deliver only once.

Use a stable operation key supplied across retries. A delivery token is unsuitable because it intentionally changes on redelivery. A payload hash alone is also a different identity policy: two legitimate operations can have the same account and delta. The build applies distinct keys separately even when their payloads match.

A stable key protects one logical effect
start / 3
Match both identity and payload before deciding to apply or skip.
A stable key protects one logical effect
Match both identity and payload before deciding to apply or skip.

The ledger stores the payload associated with each key. A repeated key with the same (account, delta) returns duplicate; a repeated key with a changed payload raises ValueError. Quietly treating conflicting reuse as success can hide a serious caller error. It also matters which fields belong to the logical operation: comparing the account while ignoring the delta would accept an altered request.

Python
balances = {}
seen = {}
for key, account, delta in [("k1", "a", 7), ("k1", "a", 7), ("k2", "a", 7)]:
    if key not in seen:
        balances[account] = balances.get(account, 0) + delta
        seen[key] = (account, delta)
print(balances, len(seen))

This small loop illustrates repeated matching payloads. The graded implementation adds full input validation, conflicting-key checks and an explicit simulated failure point. Validate before looking up duplicates so an invalid flag or boolean delta cannot slip through only because the key exists. Zero and negative integer deltas remain valid; truthiness is not a substitute for type or domain validation.

Commit and dedup record belong together
start / 3
Our injected failure occurs before either change.
Commit and dedup record belong together
Our injected failure occurs before either change.

The effect and its deduplication record must share a transaction boundary in a real store. If the record is saved before a failed effect, a retry may skip required work. If the effect commits before a failed record write, a retry may repeat it. Our fail_before_commit injection raises before either change, so the exercises can test that specific logical boundary. The ordinary Python assignments afterward do not provide a durable atomic database transaction.

A duplicate still needs delivery completion handling: the consumer can acknowledge after finding the already committed operation. A conflicting key should be reported or quarantined according to application policy, rather than retried forever. Keys also need a retention policy in a real system; forgetting one while old messages can still arrive removes the deduplication guarantee for those messages. This exercise retains keys for the object's entire lifetime.

The bug hunts deliberately bypass seen-key handling or compare only part of the payload. Tests check matching replays, independent equal-payload operations, collisions, zero/negative effects, failed commits and copied snapshots. The resulting ledger is a deterministic teaching model. It does not deliver exactly-once messages, survive process loss or make a payment/database operation atomic across separate services. Those boundaries must be designed and tested in the actual stack.

Watch & read

Free lectures, talks and readings that cover this skill from another angle. All resources →

Watch 20:30–33:50 and locate application work before acknowledgement. Use that interval to reason about a lost ack. The authored ledger and primary reading teach the idempotency contract.