When a reply is lost, the request may already have reached the server. Retrying can repeat an effect. Treat each new attempt as an explicit policy decision rather than a generic except-and-loop.
This lab’s report endpoint is a GET. The retry helper permits GET/HEAD methods and a small transient error set. It does not automatically repeat POST/PATCH. A real API may supply idempotency keys or stronger application guarantees; those require a separate design.
Check method eligibility, error class and remaining attempts. We retry timeout/transport errors and 429/502/503/504 statuses. Do not retry malformed JSON, the wrong content type or a schema failure: repeating bad data is not a repair strategy.
Use a valid Retry-After integer-seconds value when supplied. This small policy explicitly rejects HTTP-date values or ambiguous duplicates instead of pretending to understand them. Otherwise use .1, .2, .4… seconds. The real lab injects time.sleep; hosted tests inject a recorder.
max_attempts, wait_budget_ms = 3, 300
waited_ms = 0
for attempt in range(1,max_attempts):
delay_ms = 100 * (2 ** (attempt-1))
if waited_ms + delay_ms > wait_budget_ms:break
waited_ms += delay_ms
print("before attempt", attempt+1, "wait", delay_ms, "total", waited_ms)Keep cumulative waiting in integer milliseconds. That avoids a false budget failure caused by .1 + .2 being slightly greater than .3 in binary floating point. The delay policy uses whole milliseconds, while socket timeouts remain seconds.
Cancellation is checked before sending, after receiving, before sleeping and after sleeping. An external signal can stop a new attempt and prevent accepting a stale response. It cannot roll back an already sent operation. The sleep budget covers requested delays only, not request runtime, scheduler overhead or an overall deadline.
Tests count sends and sleeps. A POST failure must produce one send. A two-second Retry-After with a half-second budget must stop before sleeping. A signal raised during the injected wait must prevent the next request.
def eligible(method,code,status=None):
transient = code in {"transport","timeout"} or (code=="http-status" and status in {429,502,503,504})
return method in {"GET","HEAD"} and transient
print(eligible("GET","http-status",503))
print(eligible("POST","timeout"))
print(eligible("GET","schema"))Complete the decision block, then test unsafe methods, invalid JSON, Retry-After beyond budget, exact cumulative wait, attempt exhaustion and cancellation during waiting. Keep raw exception text and credentials out of output.
Next we use concurrency for independent category collections. Pages within one opaque cursor chain still have to be fetched sequentially.