The old joke says there are two hard things in computer science: cache invalidation, naming things, and off-by-one errors. It endures because cache invalidation really is hard — not the caching, which is easy, but knowing when the cached copy has gone stale and what to do about it. Adding a cache is a five-minute change that feels like free speed. The difficulty, and every bug, lives in the invalidation.
Why caching is easy and invalidation is not
Reading from a cache is trivial: check if the value is there, return it if so, fetch and store it if not. The hard part is that the moment you store a copy, you have two sources of truth, and they can disagree. The underlying data changes, the cache does not know, and now you are serving something wrong. Every caching pattern is really just a different answer to one question: how do we handle the copy going stale?
There is no answer that is correct everywhere, because the right amount of staleness depends entirely on the data. A user’s account balance must be current to the transaction. A “trending articles” list can be an hour old and nobody cares. The engineering is in matching the strategy to how much staleness the data can actually tolerate, and being honest that some staleness is almost always the price of the speed.
Cache-aside: the default, and its race
The most common pattern is cache-aside, also called lazy loading. The application checks the cache; on a miss it reads the database, stores the result, and returns it. On a write, it updates the database and deletes the cached entry so the next read repopulates it.
READ: value = cache.get(key)
if value is null:
value = db.read(key)
cache.set(key, value, ttl)
return value
WRITE: db.write(key, newValue)
cache.delete(key) # invalidate, let the next read refillIt is simple, it only caches what is actually read, and it survives a cache outage — a cold cache just means more database load, not wrong answers. It is the right default for most read-heavy workloads.
Its subtlety is a race condition on concurrent read and write. A read can fetch the old value from the database, and just before it writes that value into the cache, a write updates the database and deletes the (still empty) cache entry — then the read completes and stores the stale value, which now lives in the cache until its TTL expires. The pattern is mostly self-correcting because of the TTL, but if you cannot tolerate even a window of staleness, cache-aside alone is not enough. This is the same read-then-write hazard that makes concurrency hard in general, and it is why a bounded TTL is not optional insurance.
Write-through and write-behind
Write-through updates the cache and the database together on every write, keeping them in lockstep so reads are always fresh. The cost is write latency — every write now pays for two stores — and you cache data that may never be read. It suits workloads where reads dominate and staleness is genuinely unacceptable.
Write-behind (write-back) updates the cache immediately and the database asynchronously a moment later. Writes feel instant, but you have accepted a window where the cache holds data the database does not, and a crash in that window loses it. It is a performance trade with a durability cost, appropriate only when losing a few seconds of writes is survivable — and it is the wrong choice for anything you cannot afford to lose.
TTLs: the pragmatic admission that you cannot track everything
The honest truth is that precisely invalidating every cache entry the instant its underlying data changes is often more complexity than it is worth. A time-to-live is the pragmatic alternative: accept that data can be stale for up to N seconds, and let entries simply expire. You are trading exactness for enormous simplicity, and for most data that trade is correct.
The skill is choosing the TTL from how fast the data changes and how much staleness the feature tolerates — seconds for a near-live figure, hours for a leaderboard, indefinite for something immutable. A short TTL is not automatically safer; it just means more cache misses and more load. And a TTL composes well with event-driven invalidation: expire entries after N seconds as a backstop, but also actively invalidate them when you receive an event saying the data changed, so the common case is fresh and the TTL only catches what you missed. That active path pairs naturally with an event-driven architecture, where a data-change event is already flowing through the system.
The cache stampede that takes down services
The failure mode worth designing against from the start is the stampede — also called the thundering herd. A popular key expires, and in the instant before anything repopulates it, every concurrent request misses simultaneously and hammers the database with the identical expensive query at once. The cache was the only thing protecting the database, and its expiry removed that protection for everyone at the same moment.
There are two standard defences. Lock so that only the first request on a miss recomputes the value while the others wait for it, rather than all of them recomputing in parallel. Or add jitter to your TTLs, so a batch of entries created together does not all expire in the same instant — the same insight that jitter solves for synchronised retries in our retry storms guide. A stampede is the caching version of exactly that synchronisation problem, and the fix rhymes.
Cache keys are where the subtle bugs live
Most caching disasters are not invalidation failures at all — they are key-design failures, where the cache correctly returns a value that answers a different question than the one asked. The key is the identity of the cached thing, and if it omits something that changes the answer, the cache will confidently serve the wrong data.
The classic and most dangerous example is forgetting the user or tenant in the key. Cache a dashboard under the key dashboard, and the first user’s dashboard is served to everyone — a data leak, not a performance bug, and one that appears intermittently depending on who warmed the cache. The key had to include the user id, and it did not. This is the caching face of the same boundary discussed in our multi-tenant isolation guide, and it deserves the same reflexive care: anything that changes the response belongs in the key.
The same logic covers less obvious dimensions. If a response varies by language, the locale belongs in the key, or a French user gets an English cached page. If it varies by permission level, the role belongs in the key. If it varies by feature-flag state, that belongs in the key too. The discipline is to enumerate every input that changes the output and make sure each one is part of the key — because whatever you leave out is a dimension along which the cache will silently serve the wrong thing.
There is also a versioning technique worth knowing: include a version token in the key, and you can invalidate an entire class of entries instantly by bumping the version, rather than hunting down and deleting each key. Change the shape of a cached object and old entries under the previous version simply stop being read, aging out naturally, while new reads miss and repopulate under the new version. It sidesteps a whole category of “stale cache after a deploy” bugs — the entries do not need deleting because nothing looks for them any more. Key design gets far less attention than invalidation and causes at least as many incidents.
Cache the right layer, and know what breaks
Caching exists at many layers — the browser, a CDN, an application cache like Redis, the database’s own buffer pool — and the cheapest cache is the one closest to the user. A response served from a CDN never touches your infrastructure at all. Push caching outward where you can, and reserve the application cache for the dynamic, per-user data the outer layers cannot hold.
Whatever layer you choose, design for the cache being wrong, because eventually it will be. Ask what happens when a user sees stale data — for a trending list, nothing; for a permission check, a security hole. That question decides how much invalidation rigor a given cache actually needs, and it is worth answering before you add the cache, not after an incident. A cache keyed without the tenant serving one customer’s data to another is the sharp version of this, and it is why cache keys deserve the same care as any other tenant-isolation boundary. Caching is easy; being correct about staleness is the entire job.