Lesson 0002 · Storage engine

WiredTiger: how documents are stored

MongoDB's default engine gives you MVCC, durability, and space reuse — but the InnoDB way, not the Postgres way. A point-in-time snapshot per operation, a write-ahead journal plus checkpoints, and freed space it reuses on its own. No VACUUM.

~10 minWarm-up + retrieval quiz + 3-way contrastCheat sheet: here

Lesson 0001 was about the shape of data — the document. This one drops a level: how MongoDB actually puts documents on disk and keeps them safe. You have two sharp reference points already. InnoDB: a B+tree in a buffer pool, updates roughly in place, old versions in the undo log, a redo log for durability. Postgres: an append-only heap, every version inline, a background VACUUM to reclaim dead tuples, a WAL for durability. The question this lesson answers: where does MongoDB's WiredTiger engine sit between them? The short answer — and the thing to hold onto — is much closer to InnoDB.

The one idea

WiredTiger is the default storage engine, and it stores each collection and index as a B-tree — internal pages hold keys and pointers, leaf pages hold the records in sorted key order. — MongoDB Manual: "The WiredTiger storage engine is the default storage engine." · WiredTiger Architecture Guide: tables are represented as a B-Tree; internal pages store keys and references, leaf pages store keys and values in sorted order. That alone rhymes with InnoDB (a B+tree) and stands apart from the Postgres heap. But the deeper resemblance is in the three things a storage engine has to solve: concurrency, space, and durability.

Concurrency: MVCC by snapshot, at document granularity

Like both engines you know, WiredTiger uses MVCC: at the start of an operation it hands that operation a point-in-time snapshot — a consistent view of the data as of that instant. — MongoDB Manual: "WiredTiger uses MultiVersion Concurrency Control (MVCC). At the start of an operation, WiredTiger provides a point-in-time snapshot of the data to the operation. A snapshot presents a consistent view of the in-memory data." This is the same shape as the Postgres snapshot you met in Postgres 0005 and the InnoDB consistent read from MySQL 0005.

The granularity is the document: WiredTiger uses document-level concurrency control, so many clients can modify different documents of the same collection at once, coordinating with optimistic concurrency control and only intent locks above the document. — MongoDB Manual: "WiredTiger uses document-level concurrency control for write operations. As a result, multiple clients can modify different documents of a collection at the same time. … WiredTiger uses optimistic concurrency control. WiredTiger uses only intent locks at the global, database and collection levels." Read that as the direct cousin of InnoDB's row-level locking — fine-grained, not a table lock.

Space: freed room is reused automatically — no VACUUM

This is the biggest divergence from Postgres, and the place your Postgres course pays off by contrast. In Postgres, old row versions pile up inline in the heap as dead tuples, and a background VACUUM must reclaim them or the table bloats (Postgres 0002). WiredTiger does not work that way: as documents are deleted it keeps lists of the freed space and reuses it for new data on its own — there is no VACUUM to schedule. — MongoDB Manual: "The WiredTiger storage engine maintains lists of empty records in data files as it deletes documents. This space can be reused by WiredTiger…"

Durability: checkpoint + journal — the same WAL story

Nothing here should surprise you after Postgres 0009 and MySQL 0007. Two mechanisms cooperate. First, every 60 seconds WiredTiger writes a full, consistent checkpoint of a snapshot to disk — a recovery point. — MongoDB Manual: "MongoDB configures WiredTiger to create checkpoints… writing the snapshot data to disk at intervals of 60 seconds. … checkpoints can act as recovery points." Second, between checkpoints, a write-ahead journal records changes so nothing committed is lost if the server dies mid-interval. — MongoDB Manual: "To provide durability in the event of a failure, MongoDB uses write ahead logging to on-disk journal files. … if MongoDB exits unexpectedly in between checkpoints, journaling is required to recover information that occurred after the last checkpoint."

The journal is flushed on a tight cadence — every 100 ms, or immediately for a write that asked for it (j: true), or when a 100 MB journal file fills. — MongoDB Manual: journal records are synced "At every 100 milliseconds," on j: true, and when WiredTiger creates a new (~100 MB) journal file. And recovery is the WAL algorithm you already know: find the last checkpoint id, then replay the journal records written since it. — MongoDB Manual: recovery "Looks in the data files to find the identifier of the last checkpoint… Searches in the journal files for the record that matches… Apply the operations in the journal files since the last checkpoint." This is redo-style roll-forward — one log, exactly like the Postgres WAL and InnoDB's redo log.

Memory: the WiredTiger cache is the buffer pool

WiredTiger keeps the working set in an internal cache sized, by default, to the larger of 50% of (RAM − 1 GB) or 256 MB — and it also leans on the OS filesystem cache. — MongoDB Manual: "the WiredTiger internal cache size is the larger of either: 50% of (RAM - 1GB), or 0.256 GB." MongoDB "utilizes both the WiredTiger internal cache and the filesystem cache." The internal cache holds data uncompressed (ready to work on) while the filesystem cache mirrors the compressed on-disk blocks. Think of the internal cache as InnoDB's buffer pool — the hot pages in RAM.

A tax neither relational engine charges by default: compression

On disk, WiredTiger compresses by default — snappy block compression for most collections and prefix compression for indexes (zstd for time-series). — MongoDB Manual: "By default, WiredTiger uses block compression with the snappy compression library for most collections, and prefix compression for all indexes." So a MongoDB collection is typically smaller on disk than the same data would be in a default Postgres or InnoDB table — a real, free operational difference to remember.

Three engines, one table

Check yourself

Answer from memory — the effortful recall is what builds retention. Feedback is immediate.

db.serverStatus().storageEngine        // { name: "wiredTiger", ... }
db.serverStatus().wiredTiger.cache["bytes currently in the cache"]
db.serverStatus().wiredTiger.cache["maximum bytes configured"]  // ≈ 50% of (RAM − 1GB)
db.stats()                             // storageSize vs dataSize — compression + free space
db.runCommand({ compact: "yourColl" }) // the "VACUUM FULL": return freed space to the OS