An index tells us which documents match. Ranking answers a different question: which matches should a learner see first? A useful baseline makes its score explicit enough to compute by hand, test, and compare with later changes. Our score combines a term’s share of a document with a rarity weight across the corpus.
For each distinct query term, compute tf = count / document_length and rarity = 1 + ln((N+1)/(df+1)). Add tf × rarity to every containing document. N includes empty documents; df counts the documents in that term’s posting. This is one deliberately stated TF–IDF-style convention. It is not BM25, cosine-normalized TF–IDF or a promise of good relevance for every collection.
Suppose red appears once in both a two-token document and a four-token document. Its normalized frequency is 0.5 versus 0.25. The rarity multiplier is the same for both, so the shorter document scores higher for that term. A rare query term receives a larger multiplier than a common term, but a document still needs an occurrence to receive any contribution.
import math
rarity = 1 + math.log((3 + 1) / (1 + 1))
print(round(rarity, 3))
print(round((1 / 2) * rarity, 3), round((1 / 4) * rarity, 3))The query uses distinct terms on purpose. A user repeating red red should not double every red score in this course. Document occurrences remain counted; only query repetition is collapsed. Sorting query terms also fixes the accumulation order for floating-point sums.
After accumulating scores, sort by decreasing score and increasing ID. Do not round before ranking: two close but distinct scores may round to the same display value. Unknown terms add no matching documents. Empty queries, empty indexes and k=0 produce no hits, while invalid query/k types still fail the stated boundary checks.
hits = [('b', 1.0), ('a', 1.0), ('z', 2.0)]
ordered = sorted(hits, key=lambda hit: (-hit[1], hit[0]))
print([doc_id for doc_id, score in ordered[:2]])Test unequal document lengths, a rarity weight that includes an empty document, multiple query terms, repeated query terms and exact ties. Keep the input index unchanged. Ranking scores are comparative numbers under this formula, not probabilities or confidence estimates. The evaluation lesson will check whether the resulting order is useful against independently supplied relevance labels.