Extract anchors with structured HTML parsing

A hyperlink has two useful pieces of evidence: a destination reference in href, and a label in the element’s content. Those pieces have different boundaries. In <a href="guide.html">Read <strong>the guide</strong></a>, the attribute supplies guide.html; the label spans two text callbacks and a nested element. Taking only the first text callback loses information. Taking everything after the start tag until the next closing tag stops at </strong> too early.

Use parser state to express the question. When an a start event arrives, save its href and reset a list of chunks. While the anchor is active, append data. Ignore nested inline start/end events. At the matching a end event, join the chunks, apply the label policy and append a (href, label) tuple. Reset state before the next anchor. Comments contribute neither destination nor label text.

The fixtures use nonnested anchors, unique attribute names and ordinary inline children. They exclude script/style content within anchors. This keeps one active anchor sufficient. HTMLParser does not validate those restrictions or repair arbitrary broken HTML into a browser DOM. The build retains completed anchors only; an unfinished anchor at end of input produces no record. The later inventory lessons need only href start events, so their deliberately different contract can inventory an unfinished source anchor too.

from html.parser import HTMLParser

class Hrefs(HTMLParser):
    def handle_starttag(self, tag, attrs):
        if tag == "a":
            href = dict(attrs).get("href")
            if href is not None:
                print(repr(href))

parser = Hrefs()
parser.feed('<a>name</a><a href="">self</a><a href>unset</a>')
parser.close()

Missing, empty and valueless are three source cases. A missing attribute is absent from the pairs; a valueless attribute has value None; an explicitly empty attribute has value "". This course records the empty string because resolving it against a document URL has a meaningful result. A truthiness test would incorrectly discard it. The parser also decodes & in href values, so extraction returns the actual reference rather than its HTML serialization.

Define whitespace before writing tests. Our label rule concatenates chunks with no added separators, then uses " ".join(text.split()) to collapse whitespace runs and trim ends. <a>one<em>two</em></a> therefore labels onetwo, while an actual space before the em gives one two. We do not infer layout spacing, hidden CSS content, image alt text or an accessible name. An image-only anchor has an empty extracted text label.

chunks = [" Read ", "the", "\n guide "]
print(" ".join("".join(chunks).split()))
print("".join(["one", "two"]))

Build the extractor and test adjacent anchors, uppercase attribute names, entities, empty hrefs, inline nesting and an unfinished anchor. Preserve source order and duplicates: repeated occurrences may later matter to a report. Keep relative references unchanged at this stage. Resolving a URL requires its document base, and extraction alone has not supplied that policy.