Blocks tell us where inline parsing is allowed. Now scan each ordinary source line into tokens. Tokens are data: text, code, emphasis or link, plus source location. Do not emit HTML in the scanner.
Our grammar stays deliberately small. Backtick code is literal. A single-star emphasis span contains plain text. A link has a plain label and a destination without unencoded parentheses. Nesting, images and double-star strong syntax are unsupported. A literal delimiter can be displayed inside code where the subset permits it.
Accumulate ordinary characters into a buffer. When a delimiter arrives, flush that run as one text token, parse the construct and advance beyond its closer. This avoids emitting one token per plain character and makes the structure readable.
A code span can contain <tag>, stars or brackets literally. Emphasis and link labels do not recursively parse markup in this subset. Reject unsupported nesting instead of silently producing a different document.
text = "A `x < y` page"
start = text.index("`")
end = text.index("`",start+1)
print(("text",text[:start]))
print(("code",text[start+1:end]))
print(("text",text[end+1:]))A line may start after a heading or list prefix. Add the supplied content-start column to the token’s local character index. Keep the line number unchanged. This makes an unclosed marker in the third line of a paragraph point to that line, not an invented column in a joined string.
A link token keeps the destination as source data. The string javascript:evil can be parsed as a destination while remaining forbidden to render. Parsing syntax does not confer permission. The next lesson will validate the destination before inserting an escaped attribute.
The scanner never evaluates inline code, opens a URL or reads a file. “Code” names a display token. It remains source text all the way to the page.
line = "See [docs](https://example.test)"
content_column = 3
opener = line.index("[")
print("local index",opener)
print("source column",content_column+opener)
print("destination is data",line[line.index("](")+2:-1])Complete the scanner and test mixed token kinds, a Unicode prefix, content-column offsets, literal code and unsupported syntax. Verify link destination text survives intact, including one that the renderer will later reject.
Next we turn these tokens into controlled HTML using text escaping and a separate link policy.