The machine that forgets

The quote is Phil Karlton’s and everybody knows it: there are only two hard things in computer science, cache invalidation and naming things. It is repeated as a wry aside, usually by someone about to add a cache. It deserves to be read as a warning.

A cache is a second copy of the truth that has agreed, in advance, to be wrong sometimes. Every caching decision is therefore a decision about how wrong you are willing to be, for how long, and about what. Teams that write that sentence down have far fewer incidents than teams that do not, and the difference is not intelligence. It is that the second group never explicitly chose.

Three ways it goes wrong

• The stampede. The entry expires, four hundred requests miss simultaneously, and all four hundred go to the database that the cache existed to protect.
• The zombie. An entry is invalidated in one region and not another, and for six hours half your users see a price that no longer exists.
• The silent success. The cache is doing nothing at all — the hit rate has been two per cent since a key format changed in March — and nobody notices, because a useless cache looks exactly like a working one from the outside.

The third is the most common and the least discussed. There is no alert for it, because nothing is failing. Latency is fine. Error rates are fine. You are simply paying for a Redis cluster that returns misses.

Nobody pages you when the cache stops working. It just quietly becomes a very expensive way to add a network hop.

What actually helps

The interventions that work are unglamorous. Put the hit rate on the same dashboard as the error rate, so a collapse is visible next to the thing it will eventually cause. Version your key format, so a change produces a clean miss rather than a subtle mismatch. And treat time-to-live as a product decision rather than an engineering one — the question “how stale may this be?” has an owner, and it is usually not the person writing the code.

const key = `v3:price:${sku}:${currency}`

// Bumping v3 to v4 invalidates the whole namespace atomically.
// No scan, no delete storm, no partially-migrated key space.

None of that makes invalidation easy. It makes it visible, which is the most anyone has managed in forty years. The machine will still forget. The goal is only that you find out before your users do.