Lesson 0006 — Schema design: embed vs reference

Lesson 0001 gave you the instinct — data accessed together is stored together. This lesson turns that instinct into a decision framework: three questions, three cardinality tiers, and the anti-patterns that cost you most at production scale.

Warm-up — apply lesson 0005 first (cold, from memory)

A pipeline with four $lookup stages is slow. What does that diagnose about the schema?

The MongoDB Manual's own guidance: "If you are using $lookup frequently, consider whether you can restructure your schema to use embedded documents." Heavy $lookup is the document model signaling that the data belongs in one document, not two collections. That's the 0001 instinct: accessed together → stored together.

You write the pipeline [ { $sort: { ts: -1 } }, { $match: { status: "active" } } ]. Which order does MongoDB actually execute it?

The aggregation optimizer rewrites this: $match moves ahead of $sort so it can be served by an index on `status`, reducing the set $sort has to order. Written order is a suggestion; the optimizer owns the actual sequence — predicate pushdown, same as a SQL planner moving WHERE before ORDER BY.

$unwind on an array field with 50 elements produces:

$unwind explodes an array into one document per element — fanout. All 50 documents carry the parent's other fields plus one array value each. The analog is SQL's UNNEST. Later stages ($group, $match) then operate on individual elements.

In a relational database, normalize by default is almost always right — the query planner joins for free and foreign keys enforce consistency. MongoDB's planner has no join optimizer; a $lookup is always an extra collection scan (or index walk) per document that comes through. The schema has to carry the join work up front, at design time, by deciding which data to co-locate in one document. That decision is the subject of this lesson.

Three questions — ask them in order

Decision checklist

Access pattern: Is this data always read or written together with the parent, or sometimes/never? → Embed if always; reference if often independent.
Cardinality: How many related items can there be — a handful, thousands, or millions? → Embed if bounded and small; reference if unbounded or large.
Update independence: Is this data updated on its own, independent of the parent? → Reference if yes — embedding means every independent update rewrites the whole parent document.

The 16 MB document limit enforces question 2 at the hardware level: an unbounded embedded array will eventually crash a write. Questions 1 and 3 are softer but matter at scale — a 500 KB document that grows by embedding everything strains the WiredTiger cache and costs more network bandwidth per read.

Three tiers by cardinality

One-to-few → embed

The sub-documents are small, bounded, and always accessed with the parent.

Examples: a person's mailing addresses (1–5), a line item list on an order (1–50), the author name and short bio on a blog post.

All three questions point the same way: always accessed together, small set, not updated in isolation. Embed them. You get one read instead of a parent read + a join, and you get single-document atomicity for free.

— MongoDB Manual: "Embedding provides better performance for read operations… In addition, embedded data models allow applications to update related data in a single atomic operation."

One-to-many → reference (usually)

The sub-documents are numerous, may grow significantly, or are sometimes accessed on their own.

Example: a product with hundreds of review documents. You read a product page far more often than you need all reviews — the review list is paginated, filtered, sorted. Reviews are also accessed on their own (moderation queue, author history). Embedding all reviews in the product document would bloat every product read and eventually exceed 16 MB.

Reference here: each review document holds a productId field (or the product document holds an array of review _id values — bounded if you cap it). The cost is a second query or a $lookup; the gain is a product document that stays small and reviews that can be queried independently. — MongoDB Manual: "Normalized data models describe relationships using references between documents… normalized models have the following properties: large hierarchical data sets or documents that frequently change."

One-to-squillions → reference on the child side

The sub-documents are in the millions. An IoT device with millions of sensor readings; a user account with millions of activity events; a microservice emitting millions of log lines.

The naive reference approach — an array of reading _id values on the device document — is itself an unbounded array anti-pattern: the device document grows forever. The fix is to flip the reference direction: store the parent's _id on each child document instead. Each log entry carries a deviceId field. The device document stays small; querying all readings for a device becomes an indexed range scan on the readings collection.

— MongoDB Blog: "For one to squillions relationships, use a parent-reference in the child document to avoid growing an array on the parent without bound."

Atomicity is a schema argument

This is easy to forget: the embed-vs-reference decision changes what is and isn't atomic, without any explicit transaction.

When you embed related data — say, an author's name and bio directly in the blog post document — updating the post's title and the author's bio at the same time is a single document write. WiredTiger writes it atomically: either both fields update or neither does.

When you reference — keep the author in a separate authors collection — you now need two writes: one to update the post, one to update the author. If the process crashes between them, you have an inconsistency. Correcting that requires a multi-document transaction, which adds coordination overhead across two collections.

The rule: if two pieces of data must always be consistent with each other, consider whether embedding them in one document lets the engine enforce that consistency for you — for free, without a transaction.

This does not mean "embed everything." The cardinality questions still apply. But for the one-to-few cases where embedding is appropriate, atomicity is a second, independent argument in its favor — not just read locality.

Anti-patterns

The subset pattern — naming what you already intuited

One-to-many at the boundary: the parent usually needs some of the child data — the five most recent reviews, the top three comments — but rarely all of it. The subset pattern handles this: embed the hot slice in the parent document; keep the full set in its own collection.

A product document embeds the 5 most recent reviews as a recentReviews array. All reviews also live in a reviews collection with a productId reference. The product page renders instantly from one read (the embedded subset handles the above-the-fold case); the "see all 2,400 reviews" link queries the separate collection.

The cost: two writes on every new review (the reviews collection + updating the parent's embedded array) and a risk of the two going stale relative to each other. The gain: the hot path never pays for a $lookup. Worthwhile when the read-to-write ratio is high.

Relational contrast — why the instincts differ

Question	Relational (Postgres / MySQL)	MongoDB
Default modeling move	Normalize — split into tables, join at query time	Embed — co-locate access-correlated data
Cost of a join	Low to zero — query planner has join order optimization, index nested loops, hash join, sort-merge; a PK lookup is 2–3 I/Os	Always non-zero — $lookup is a separate collection pass per document; no join order optimizer across collections
Atomicity unit	Transaction — any `BEGIN … COMMIT` is atomic across tables	Document — one document write is atomic without a transaction; cross-document needs an explicit transaction
Schema enforcement	DDL — column types, NOT NULL, FK constraints enforced by the engine	Application-level or optional validator; documents in one collection can differ in shape
Update cardinality risk	Low — updating one row doesn't touch related rows (FK, not copy)	Embed → update recopies nested data; reference → two writes with no automatic consistency
"Normalize everything" failure mode	None (it's the right default)	$lookup-heavy pipelines — read locality destroyed, pipeline optimizer can't help much

An IoT fleet: one device, potentially millions of sensor readings. Where does the parent reference belong?

One-to-squillions: storing reading _ids in an array on the device document creates an unbounded array that eventually hits the 16 MB limit. Flip the reference — each reading carries a deviceId. The device document stays small; querying all readings for a device is an indexed range scan on the readings collection. Junction collections are a SQL pattern; MongoDB doesn't need them.

Your app updates both a post's title and its embedded author's bio in one call. That update is atomic because:

Single-document atomicity (0001): writes to any fields within one document, including nested fields and arrays, are all-or-nothing. No transaction needed. If the author were a separate referenced document, you'd need two writes, and inconsistency is possible without an explicit transaction.

A product document embeds the 5 most recent reviews; all reviews also live in a separate reviews collection. This is:

The subset pattern deliberately duplicates a slice of high-traffic data into the parent document so the hot read path (render the product page) needs no $lookup. Full review lists come from the separate collection. The duplication is the point — you trade write-time cost (update two places) for read-time speed (one document fetch).

Why is storing event-log entries in an array on a parent document an anti-pattern?

An unbounded array is the canonical MongoDB schema anti-pattern. The document grows monotonically — every new event extends the array. It will hit the 16 MB cap on a write, and well before that it balloons every read of the parent document. Reference on the child side (each event carries a parentId) instead.

Treating MongoDB like a relational database — referencing every entity — leads to:

MongoDB has no FK constraint enforcement and no join optimizer. Referencing everything means the pipeline must $lookup to reassemble what should be co-located. The document model's promise — "one read, no join" — is entirely voided. The right default is embed, with reference reserved for the cases where cardinality or update independence demand it.

Optional — when you have a mongosh instance: insert a post document with an embedded author sub-document and a small tags array. Then update a tag in one operation and observe that the entire document (including the author) is rewritten atomically — check with db.posts.findOne() before and after. Then try to embed 5,000 dummy comments in a loop and watch the document grow toward the limit.