Lesson 0004 · Caching patterns

Cache-aside & the stampede problem

The pattern behind almost every Redis cache — and the thundering herd that TTLs quietly set up.

~11 minRetrieval quiz at the endCheat sheet: here

You now know (Lesson 0003) that a TTL makes a key vanish at a moment in time. This lesson builds the caching pattern that relies on that — cache-aside — and then confronts the failure it sets up: the instant a hot key expires, thousands of requests miss at once and stampede your database. Getting cache-aside right is table stakes; knowing how to tame the stampede is what separates "I use Redis" from "I run Redis in production."

Part 1 — Cache-aside (lazy loading)

Cache-aside puts the application in charge: it consults the cache, and only touches the database on a miss. The read path is three steps — AWS — Database Caching Strategies (Caching patterns):

1. Check the cache. Data present → cache hit, return it.
2. On a miss, query the database.
3. Populate the cache with what you fetched (with a TTL), then return it.

# cache-aside read
val = GET key
if val is nil:                 # cache miss
    val = db.query(...)        # go to the source of truth
    SET key val EX 300         # populate + TTL (lazy load)
return val

Because the cache fills only on demand, it holds just what the app actually requests — cheap and self-trimming. The cost: every miss pays one slow DB round-trip, and the data can go stale between the DB changing and the TTL expiring. On writes, cache-aside is usually paired with either write-through (update DB, then immediately update the cache) or simply invalidation (DEL key on write, let the next read repopulate). — AWS whitepaper

AWS documents cache-aside and write-through in detail; read-through and write-behind are the standard industry terms for the other two rows.

Part 2 — The stampede TTLs set up

A cache stampede (a.k.a. thundering herd or dogpile) is a cascading failure: a popular key expires, and every concurrent request that was being served from it now misses simultaneously — so they all run step 2 (query the DB) at the same instant. The database, which the cache existed to protect, takes the full unbuffered load. — Cache stampede (Vattani et al., VLDB 2015)

Part 3 — Four ways to tame it

1. TTL jitter — always do this

Add a random spread to every TTL so keys don't expire in lockstep: EX (300 + rand(0..60)). Cheapest possible fix, kills the synchronized-expiry variant outright. — AWS ElastiCache — add jitter to TTLs

2. Single-flight lock — one recompute, not ten thousand

On a miss, have exactly one request rebuild the value while the rest wait briefly or serve stale. Redis gives you the atomic primitive for free (recall from Lesson 0001: commands are atomic): SET lock:key token NX EX 10 succeeds for only one caller. — Redis Docs — SET NX EX

val = GET key
if val is nil:
    if SET lock:key <token> NX EX 10:   # exactly one winner
        val = db.query(...)
        SET key val EX (300 + jitter)
        # release only our own lock (see warning)
    else:
        sleep a little / serve stale, then re-GET
return val

3. Probabilistic early recomputation (XFetch)

Instead of waiting for expiry, let each reader probabilistically refresh the key before it dies, with the probability rising as the TTL approaches. One lucky request rebuilds early; everyone else keeps getting a cache hit — the herd never forms. This is the Vattani et al. result; the trigger recomputes when now − delta·β·ln(rand) ≥ expiry, where delta is the last recompute duration and β tunes eagerness. — "Optimal Probabilistic Cache Stampede Prevention", VLDB 2015

4. Stale-while-revalidate

Serve the last-known value while one background refresh runs — no reader ever blocks on the DB. Keep the value under a longer "hard" TTL than its "fresh" window; past freshness, return stale and trigger a single async rebuild. This is the same idea RFC 5861 standardized for HTTP caches. — RFC 5861 — stale-while-revalidate

Check yourself

Answer from memory — effortful recall builds retention. Two questions revisit earlier lessons on purpose (spacing).

Hands-on: cause a stampede, then stop it

You need a local Redis and redis-cli. We'll fake a "slow DB" with a sleep.

1 · The single-flight lock in raw commands

# two terminals racing to rebuild "report" — only one should win the lock
redis-cli SET lock:report t1 NX EX 10     # terminal A -> OK
redis-cli SET lock:report t2 NX EX 10     # terminal B -> (nil): blocked
# A does the slow work, writes the value with jitter, then releases safely:
redis-cli SET report "…computed…" EX 305
redis-cli EVAL "if redis.call('get',KEYS[1])==ARGV[1] then return redis.call('del',KEYS[1]) else return 0 end" 1 lock:report t1

2 · See jitter desynchronize expiry

redis-cli EVAL "for i=1,5 do redis.call('set','p:'..i,i,'EX',60+math.random(0,30)) end" 0
redis-cli TTL p:1
redis-cli TTL p:2                          # different — no lockstep expiry

Clean up with redis-cli FLUSHALL. Bring your lock-race reasoning to your teacher.