Reference · Cheat sheet

BM25 scoring

The compressed essence. Print it, pin it.

From lesson 0002See also glossary

score(t,D) = IDF(t) · f(t,D) · (k1 + 1) f(t,D) + k1 · (1 − b + b · |D|avgdl) summed over query terms

IDF(t) = ln(1 + (N − n + 0.5) / (n + 0.5))

Three intuitions

Part	Says	In the formula
IDF	Rare terms matter more	`ln(1 + (N−n+0.5)/(n+0.5))`
TF saturation	Repetition helps, with diminishing returns	fraction → ceiling `(k1+1)`
Length norm.	Short fields count more	`b · \|D\|/avgdl` in denominator

The two knobs

Knob	Default	Controls	Extremes
`k1`	1.2	TF saturation rate	`0` = ignore frequency; higher = saturate slower
`b`	0.75	Length normalization	`0` = ignore length; `1` = full penalty

Gotchas

Per-shard IDF. N/n are shard-local → scores can differ by shard. Use search_type=dfs_query_then_fetch for global stats.
Not a percentage. _score is unbounded and only comparable within one query + index.
Default since ES 5.0. BM25 replaced classic TF/IDF (classic similarity).
Query context only. filter / constant_score skip BM25 entirely.

Debugging

GET /index/_explain/<id>        // full BM25 breakdown for one doc
{ "query": { "match": { "title": "tuning" } } }

GET /index/_search?explain=true  // per-hit explanation inline

← lesson 0002 · Source: Connelly — Practical BM25, Part 2