Lesson 0001 · Relevance & Query DSL

Query context vs Filter context

One distinction that shows up in almost every real query — and almost every interview.

~7 minRetrieval quiz at the endCheat sheet: here

You already write Elasticsearch queries. This lesson sharpens one thing you half-know into something you can defend: every clause you write runs in one of two contexts, and choosing the right one is simultaneously a relevance decision and a performance decision. Interviewers probe this precisely because it reveals whether you understand what the engine is actually doing.

The one idea

Every clause answers one of two questions:

Context	Question it asks	Output
Query context	“How well does this document match?”	A relevance `_score`
Filter context	“Does this document match — yes or no?”	No score. Just inclusion.

Query context computes a _score (default: BM25 — see Lesson 0002). Filter context skips scoring entirely: a document is either in the set or it isn't. — Elastic Docs: Query and filter context

Where each clause lives

The bool query is where this becomes concrete. Its four keys split cleanly across the two contexts:

bool key	Context	Must match?	Affects score?
`must`	query	Yes	Yes
`should`	query	Optional*	Yes
`filter`	filter	Yes	No
`must_not`	filter	Must not	No

* should is where people trip. See “The should trap” below.

The key insight

must and filter do the same matching — both require the clause to match. The only difference is that must spends effort computing a score and filter doesn't. So if a clause shouldn't influence ranking, putting it in must is pure waste.

Why it matters — two payoffs at once

1. Relevance: keep noise out of the score

Constraints like status = published, a date range, or a permission check are binary facts, not signals of relevance. If you score them, an exact-match term on a rare status value can distort ranking. Filtering them keeps _score reflecting only what the user actually searched for.

2. Performance: filters are cached and skip scoring

Filter-context clauses are eligible for the node query cache: Elasticsearch remembers which documents matched as a compact bitset, keyed by the filter. Reuse that filter on the next request and it's a cache hit — no re-evaluation, no scoring math. — Elastic Docs

That's why the standard pattern is: full-text in must, constraints in filter.

The canonical shape

GET /articles/_search
{
  "query": {
    "bool": {
      "must":   { "match": { "title": "elasticsearch tuning" } },   // scored (relevance)
      "filter": [
        { "term":  { "status": "published" } },                     // yes/no, cacheable
        { "range": { "published_at": { "gte": "2024-01-01" } } }
      ],
      "must_not": { "term": { "archived": true } }                  // exclusion, no score
    }
  }
}

The match in must ranks the results. Everything in filter and must_not just carves the candidate set — cheaply and cache-friendly.

Interview-grade nuances

The should trap

What does should mean depends on its neighbors. With no must/filter, at least one should must match (minimum_should_match defaults to 1) — it behaves like OR. Add a must or filter, and should becomes optional (default 0): a pure relevance boost for docs that happen to match.

constant_score

Need a clause to filter (match/no-match, cached) but still return a score? Wrap it in constant_score — every match gets the same fixed score (default 1.0), with none of the BM25 cost.

Watch out

Filters change which documents come back — they are not "free extras." Moving a clause from must to filter never changes the result set (matching is identical); it only removes that clause's contribution to _score and makes it cacheable.

Check yourself

Answer from memory — the effortful recall is what builds retention. Feedback is immediate.

Which bool clause runs in filter context (no score computed)?

filter and must_not are filter context. must and should are query context and contribute to _score.

A status: active constraint must not influence ranking. Where does it belong?

Binary constraints go in filter: same matching as must, but no wasted scoring and it becomes cacheable.

Why are filter-context clauses often faster on a repeated query?

The node query cache stores which docs matched a reused filter as a bitset — a cache hit avoids re-evaluation and scoring entirely.

A bool has one must clause and two should clauses. Default minimum_should_match is:

Because a must is present, should becomes optional (default 0) — it only boosts the score of docs that happen to match.

Moving a matching clause from must to filter changes what?

Matching is identical, so the result set is unchanged. What changes: the clause no longer adds to _score, and it becomes cacheable.

Hands-on: prove it on your cluster

Recall is one thing; watching the numbers move is another. Run this against a live cluster (Kibana Dev Tools shown; curl works too). The goal is to see that must → filter changes the score but not the result set.

1 · Seed a tiny index

PUT /lab_articles/_bulk
{"index":{"_id":1}}
{"title":"elasticsearch tuning guide","status":"published"}
{"index":{"_id":2}}
{"title":"elasticsearch relevance tuning tuning","status":"published"}
{"index":{"_id":3}}
{"title":"postgres tuning","status":"published"}
{"index":{"_id":4}}
{"title":"elasticsearch tuning","status":"draft"}

2 · Two queries — predict before you run the second

// A — constraint in must (scored)
GET /lab_articles/_search
{ "query": { "bool": { "must": [
  { "match": { "title": "elasticsearch tuning" } },
  { "term":  { "status": "published" } }
]}}}

// B — constraint in filter (not scored)
GET /lab_articles/_search
{ "query": { "bool": {
  "must":   { "match": { "title": "elasticsearch tuning" } },
  "filter": { "term": { "status": "published" } }
}}}

Predict first

Before running B: will the set of returned docs change? Will the _score values change? Commit to an answer, then run it.

What you should observe

Same hits, same order in A and B — docs 1, 2, 3 (doc 4 is draft, excluded either way). Matching is identical.

Every _score is higher in A than in B, because the status term adds its own contribution in query context. In B that clause is a silent yes/no. That gap is the lesson, made numeric.

3 · Two quick confirmations

// must_not is filter context → excludes, never scores
GET /lab_articles/_search
{ "query": { "bool": {
  "must":     { "match": { "title": "tuning" } },
  "must_not": { "match": { "title": "postgres" } }
}}}

// constant_score → matches like a filter, flat score of 1.0 (no BM25)
GET /lab_articles/_search
{ "query": { "constant_score": { "filter": { "match": { "title": "elasticsearch" } } } } }

Expect every hit in the last query to have exactly "_score": 1.0. When you're done, DELETE /lab_articles to clean up — and bring your A-vs-B _score numbers to your teacher if anything surprised you.

Primary source — read this next

Elastic Docs — Query and filter context. Short, authoritative, and the reference every one of these claims traces back to. Read it end to end; it's under 500 words.

Where this goes next

You now know whether a clause is scored. Lesson 0002 — BM25 scoring covers how that score is actually computed: the BM25 formula, term-frequency saturation, and field-length normalization. That's the other half of every relevance conversation.

Lesson 0002 — BM25 scoring → Cheat sheet → Glossary → All lessons

I'm your teacher for this — if anything here is fuzzy (why filters cache, what dfs_query_then_fetch has to do with scoring, how this looks in your own index), just ask. Bring me a real query and we'll dissect it.