Pagination, opaque cursors & termination

An API often splits a collection into pages. Pagination is a continuation protocol: the response tells the client how to ask for more. Some endpoints use numeric offsets, others return opaque cursor strings, and others provide next links in metadata. Read the endpoint's contract instead of assuming that every query parameter named page can be incremented safely.

Our offline protocol stores decoded page objects under cursor keys. Each page has items and optional next. A cursor such as z/9?x=1 is an identity token, not a number, pathname or permission to fetch an arbitrary URL. Follow it exactly in the supplied fixture mapping. The grading does no network I/O and does not implement GitHub's Link-header protocol or any other provider API.

An empty intermediate page can still contain next. Terminate only when next is absent or null. Preserve page order and item order, including duplicates. Deduplicating by an item ID would be a different application transformation and could discard meaningful repeated observations.

pages = {
    "start": {"items": [], "next": "z"},
    "z": {"items": [{"id": 2}, {"id": 1}], "next": "a"},
    "a": {"items": [{"id": 2}], "next": None},
}
cursor = "start"
order = []
while cursor is not None:
    order.append(cursor)
    cursor = pages[cursor].get("next")
print(order)  # ['start', 'z', 'a']; not alphabetical order

Traversal needs independent termination controls. Track visited cursor identities in a set to reject cycles. Also bound total page visits, including empty pages. A long acyclic chain can still consume unlimited work without a budget. Check len(seen) >= max_pages before visiting another page: equality already means every allowed visit has been used.

A missing fixture, malformed visited page, cycle or exhausted budget raises ValueError. The contract never returns partial output labeled as complete. Validate visited pages when reached; an unused malformed mapping entry cannot invalidate a successful independent chain. Exactly the budget is allowed when the last visited page terminates.

The result is a new list of shallow-copied item dictionaries. Copying only the outer list would still share each original dictionary. dict(item) lets a caller change returned top-level fields without changing the fixture. Nested lists or objects can remain shared: that is the documented shallow-copy boundary, not a claim of deep independence.

The build tests both cursor and data invariants. Before implementation, walk an empty first page with a nonempty successor, a self-cycle, a two-page cycle, and a two-page chain with a one-page budget. Then predict whether two identical records should appear twice. The bug hunts isolate early empty-page termination and an extra visit at the limit.

Live pagination adds dataset changes, cursor expiry and consistency questions. A visit limit does not produce a stable server snapshot. Compare this local protocol with GitHub's official pagination guide: its documented next links and endpoint-specific parameters demonstrate why continuation must come from the actual API contract. The capstone separately handles retries for one page and preserves its next cursor for a caller.