Ihab Khattab

Cache Stampede: Why Your Cache Doesn’t Actually Cache

3 min read

Most caching bugs are obvious. You forget to invalidate. You cache the wrong thing. You set a TTL that’s too long.

This one is different. It hides in code that looks correct, passes every test, and only reveals itself when real users show up.

A Cache That Doesn’t Actually Cache

Here’s a function you’ve probably written before:

const cache = {};
async function getArticle(articleId) {
  if (cache[articleId]) {
    return cache[articleId];
  }
  const data = await fetch(`/api/articles/${articleId}`);
  const json = await data.json();
  cache[articleId] = json;
  return json;
}

We’re using a plain in-memory object here to keep the example focused. In a real system this would be Redis, Memcached, or whatever your stack uses, but the pattern and the bug are the same regardless of where the data lives.

The logic is straightforward: check the cache, return early if it’s there, fetch and store if it isn’t. In development, it works perfectly. In tests, it works perfectly. Under real traffic, it quietly breaks.

The bug isn’t in what the code does. It’s in what happens between the lines.

Cache Stampede: When the Herd Comes Thundering In

When JavaScript hits an await, it doesn’t just pause and wait. It yields, handing control back to the event loop and letting other code run while the async operation is pending.

That gap is where things fall apart.

Between the cache miss check and the cache write, there’s an entire network round-trip worth of time where other callers can come in. And when they do, they check the cache, find it empty (because nothing has been written yet), and fire their own fetch.

For a quiet endpoint with occasional traffic, this is invisible. For a popular article under load, say one that just got shared and is pulling in 50 simultaneous readers, you’re not making 1 API call. You’re making 50 identical ones.

This problem has a name: cache stampede, sometimes called a thundering herd. The cache exists, it’s wired up correctly, and it’s completely failing its job.

Here’s what that looks like:

Every caller races through the same open window. The API takes a beating it never needed to.

The Fix: Cache the Promise

The root cause is caching the result of the fetch instead of the fetch itself. By the time the result exists, the window is already open.

The fix is to cache the Promise immediately, before any yield point:

const cache = {};
function getArticle(articleId) {
  if (cache[articleId]) {
    return cache[articleId];
  }
  cache[articleId] = fetch(`/api/articles/${articleId}`)
    .then((res) => res.json())
    .catch((err) => {
      delete cache[articleId];
      throw err;
    });
  return cache[articleId];
}

Now the cache is written synchronously, before any yield happens. Every concurrent caller that comes in after the first one finds the Promise already there and awaits the same result. The fetch fires exactly once regardless of how many callers are waiting.

One fetch. Every caller gets the result. The herd never forms.

One thing this leaves unaddressed is expiry. The Promise stays cached indefinitely, fine for a demo, but in production you’d want a TTL: either a setTimeout that deletes the entry after a set duration, or a caching library that handles expiry for you.

This specific technique, caching the Promise itself, applies to in-memory caches. With Redis or a distributed cache, you’d tackle this differently, typically with a lock or a “set if not exists” operation that lets only one caller through while the rest wait. The underlying problem is identical; only the tool for solving it changes.

The Detail That’s Easy to Miss

Notice the .catch block. It’s not just error handling, it’s cache hygiene.

If you cache a Promise and that Promise rejects, the rejected Promise stays in your cache. Every future caller will get the same failed result, forever, until you restart. The .catch evicts the entry on failure so the next caller gets a clean attempt instead of a permanent error.

One line. Easy to skip. Critical to include.

The Broader Lesson

In synchronous code, a cache miss is a single moment: you check, you miss, you fetch, you write. No one else can interrupt that sequence.

In async code, a cache miss is a window. Between the check and the write, the event loop is open for business and other callers are free to walk in.

Once you see it, you’ll notice the pattern everywhere, not just in caches, but in any async code where you check state, do work, then update state. The check and the update are never truly atomic unless you make them so.

The fix is always some version of the same idea: don’t cache what you got. Cache your intention to get it.

← All posts