Caching: trading freshness for speed

Caching is one of the gentler ways to make a system faster. You put Redis in front of the database, a CDN in front of the app, or a memoized function in front of expensive code. Latency drops, and the source of truth gets some breathing room.

The cost of that speed is small until it isn't, and it always shows up in particular moments. A user updates their profile and sees the old name for five minutes. A hot key expires and 20,000 requests arrive at the database in the same instant. A private response is stored under a public key and handed to the wrong person. These are the moments where caching stops being a convenience and becomes something to design.

At its center, a cache is a quiet agreement: about how stale a value may be, who is allowed to share it, and how it can be rebuilt if it disappears.

Start with the stale promise

Before anything else, there is one question worth sitting with:

How wrong is this value allowed to be, and for how long?

This is the stale promise, and it varies naturally with the data. A product logo can be a day old. A user's display name can be a minute behind. A home feed can lag by a few seconds and no one minds. A checkout price should not be stale unless it is revalidated before the purchase completes. A bank balance might be cached for display, but the ledger cannot lean on it to decide whether money is allowed to move.

A cache without a named stale promise is correct only by accident. That accident usually reveals itself the day two screens quietly disagree about the same value.

The cache is a read model

A cache is a derived read model with a short memory. Because of that, the same questions from the database post still apply here:

1. What is the read path?
2. What key identifies the answer?
3. How much stale data is acceptable?
4. What writes make the answer wrong?
5. Can the answer be rebuilt?
6. What happens when many callers rebuild it at once?

The last question is the one most easily forgotten. A cache speeds up the hit path, but it also quietly shapes the miss path. If a hot key serves 10,000 QPS and its cached value expires, every one of those requests turns into a database request until something steps in to coordinate the refill.

Cache layers

Caching happens in layers, and each layer answers two questions: who shares the value, and who owns its invalidation.

A good instinct is to use the highest layer that can safely answer the request. A static image belongs on a CDN. A public product catalog page belongs on a CDN with purge or a short TTL. An authenticated account dashboard belongs in an application cache keyed by tenant and user permissions. An expensive analytics number is best precomputed, stored, and given an age.

There is a natural tension here worth noticing. A cache near the user answers in microseconds, but it is owned by that user's device or edge node, so invalidating it means reaching across many independent caches. A cache in Redis answers in a few milliseconds from one shared place, so a single delete reaches every reader at once. Closeness buys speed; centralization buys control.

Cache keys are data models

The key is the schema.

A bare key names one dimension:

profile:123

A full key names every dimension that changes the answer:

profile:v3:tenant:88:user:123:viewer:456:locale:en-US

The second key carries the tenant, the viewer, the locale, and the schema version, because each of those genuinely changes what the profile should return. Whenever the response varies by tenant, viewer permissions, locale, feature flag, or API version, that dimension belongs in the key, so each distinct answer rests in its own slot. A key that names only the user id collapses every variation of that profile into a single slot, and whichever answer happened to arrive first is then served to everyone.

Common key dimensions:

Versioned keys are the simplest deploy-time invalidation strategy:

CACHE_VERSION = 3


def profile_cache_key(tenant_id, user_id, viewer_id, locale):
    return (
        f"profile:v{CACHE_VERSION}:"
        f"tenant:{tenant_id}:"
        f"user:{user_id}:"
        f"viewer:{viewer_id}:"
        f"locale:{locale}"
    )

When the representation changes, you bump the version and let the old values expire on their own. It is simple, and it works.

TTL is a safety fuse

A TTL is a fuse. It guarantees the entry dies once the timer runs out, and nothing more. In the window before that, the entry holds whatever value it was handed at write time. If a user changes their display name and the cached profile carries a 10-minute TTL, the old name can stay visible for up to 10 minutes, until the write path invalidates the key. That window may be perfectly acceptable. The point is that it is a product decision, made on purpose rather than discovered later.

There are three basic choices:

A comfortable default is TTL plus invalidate on write. The two cover each other: the TTL catches any invalidation you miss, and invalidation spares the user from waiting out the full TTL.

Cache-aside

Cache-aside is the pattern you will reach for most often:

read:
  check cache
  if hit, return
  if miss, load from source of truth
  write cache
  return

write:
  update source of truth
  delete affected cache keys

Here is the small version:

def get_profile(user_id):
    key = f"profile:v1:user:{user_id}"

    cached = cache.get_json(key)
    if cached is not None:
        return cached

    profile = db.get_profile(user_id)
    cache.set_json(key, profile, ttl_seconds=300)
    return profile

This is enough for cold or low-traffic keys. A hot key that goes missing needs more care, so that many callers do not all rush to rebuild it at the same moment. A production version looks like this:

import random
import time


def ttl_with_jitter(base_seconds, jitter_ratio=0.15):
    spread = base_seconds * jitter_ratio
    return int(base_seconds + random.uniform(-spread, spread))


def get_profile(user_id):
    key = f"profile:v2:user:{user_id}"
    stale_key = f"{key}:stale"
    lock_key = f"lock:{key}"

    cached = cache.get_json(key)
    if cached is not None:
        return cached

    lock_acquired = cache.set_nx(lock_key, "1", ttl_seconds=10)

    if not lock_acquired:
        stale = cache.get_json(stale_key)
        if stale is not None:
            return stale

        time.sleep(0.050)
        cached = cache.get_json(key)
        if cached is not None:
            return cached

        return db.get_profile(user_id)

    try:
        profile = db.get_profile(user_id)
        cache.set_json(key, profile, ttl_seconds=ttl_with_jitter(300))
        cache.set_json(stale_key, profile, ttl_seconds=1800)
        return profile
    finally:
        cache.delete(lock_key)

This is still a simplified version. In a real Redis implementation, the lock should carry a unique token, so that one caller cannot accidentally delete another caller's lock after a timeout.

The general shape is calm and predictable:

1. One request rebuilds the missing value.
2. Other requests use stale data or wait briefly.
3. TTL jitter prevents many keys from expiring at the same second.
4. The database is the fallback, reached only when the cache cannot answer.

The stampede calculation

The arithmetic is plain, and worth looking at directly:

hot key traffic:       8,000 requests/sec
database read time:    250ms
cache entry expires:   now

concurrent DB reads = 8,000 x 0.250 = 2,000

One expired key can summon 2,000 concurrent database reads. If each query holds a connection for 250ms, the connection pool drains, unrelated endpoints begin to wait, clients retry, and the trouble spreads into the territory covered by the API reliability post. It helps to hold the distinction in mind: a five-minute TTL decides how long the hit path stays warm, while the miss path is decided entirely by what coordinates the refill at the moment of expiry.

Stampede defenses

A handful of defenses, each with its own give-and-take:

Probabilistic early refresh looks like this:

import random


def should_refresh_early(ttl_remaining, original_ttl):
    age_ratio = 1.0 - (ttl_remaining / original_ttl)
    return random.random() < max(0.0, age_ratio ** 3)

Early in the TTL, a refresh is unlikely. As the entry ages, one of the readers becomes likely to refresh it before it fully expires. The extra writes are worth it on hot keys, where a synchronized expiry would otherwise be expensive.

Write-through, write-behind, and friends

The whole family of write patterns comes down to one question: where does the acknowledgment boundary sit? That is the point in the write where the caller is told it succeeded.

For most product systems, cache-aside is a reasonable default. Write-through suits the case where read-after-write matters and the write rate stays manageable. Write-behind acknowledges the write at the cache and flushes to the database later, which opens a loss window. That window is fine for metrics, counters, or logs where the loss and replay rules are spelled out, and it is dangerous for canonical data, which is at risk for exactly the same span. Only let the cache become the single home of the truth when the system is deliberately built as an in-memory store with its own durability story.

Invalidation is part of the write path

The write path post described the flow this way:

command -> fact -> effects -> projections

Cache invalidation is one of those effects, and it deserves the same care you would give search indexing or notifications.

For a profile update:

def update_profile(user_id, display_name):
    with db.transaction() as tx:
        tx.update_profile(
            user_id=user_id,
            display_name=display_name,
        )
        tx.insert_outbox_event(
            event_type="profile.updated",
            aggregate_id=user_id,
            payload={"user_id": user_id},
        )

    return {"status": "updated"}

Then a worker invalidates derived keys:

def handle_profile_updated(event):
    user_id = event.payload["user_id"]

    cache.delete(f"profile:v2:user:{user_id}")
    cache.delete(f"profile:v2:user:{user_id}:stale")
    cache.delete(f"feed_header:v4:user:{user_id}")

Invalidation can run synchronously when freshness is critical, but with many derived keys, doing it inside the user request can turn a simple update into a slow chain of integration work. As always, the choice traces back to the freshness promise.

Deleting on writes

On a write, deleting the cached value lets the next read rebuild it cleanly from the source of truth. A single write can touch many read shapes at once: changing a display name reaches the user's profile, their posts, comment rows, notification previews, search snippets, and team member lists. Deleting the known keys is one small operation per key, and the rebuild that follows pulls the correct value through the ordinary read path. Updating each of those entries in place would instead ask the write to reproduce every read shape correctly at write time, which is far more to get right.

The honest difficulty is key discovery: keys you cannot name, you cannot invalidate precisely. That leaves three options:

1. Use shorter TTLs.
2. Use versioned namespace keys.
3. Maintain an index of dependent keys.

Namespace versioning is a practical middle ground:

def get_profile_namespace(user_id):
    version = cache.get_int(f"profile_namespace:user:{user_id}") or 1
    return f"profile:v{version}:user:{user_id}"


def invalidate_profile_namespace(user_id):
    cache.incr(f"profile_namespace:user:{user_id}")

A single write increments the namespace version. New reads compose new keys, and the old keys simply sit unused until their TTL expires them. Those old values hold their memory for the rest of their TTL, and in exchange invalidation stays as light as one increment.

What I notice

What I keep noticing is that caching bugs are usually ambiguity bugs. Everyone knows a value is cached. What no one is quite sure of is:

1. Whether the cached value is public or personalized.
2. Whether the key includes permissions.
3. Which write invalidates it.
4. Whether stale data is acceptable.
5. What happens when the cache is cold.

A cache makes the system faster by hiding work, and the same hiding makes it easy to lose track of ownership. The remedy is small and steady: write the contract down next to the cache itself.

HTTP caching is a contract too

CDN and browser caches use headers as their API.

For a public static asset:

Cache-Control: public, max-age=31536000, immutable

This works when the filename is content-hashed:

/assets/app.7f3a21c.css

If the content changes, the URL changes.

For an authenticated API response:

Cache-Control: private, max-age=30
Vary: Authorization, Accept-Language

Here private asks shared caches not to store the response, and Vary tells them which request headers change the answer.

For sensitive data:

Cache-Control: no-store

It is worth being explicit with the headers around private content. When a cache key ignores identity, one user's response can land in front of another user, and that is precisely the failure these headers exist to prevent.

Negative caching

Absence is worth caching too. If a crawler asks for the same nonexistent product id 10,000 times, the database should not have to prove that nothing is there 10,000 times over.

def get_product(product_id):
    key = f"product:v1:{product_id}"
    cached = cache.get_json(key)

    if cached == {"missing": True}:
        return None

    if cached is not None:
        return cached

    product = db.get_product(product_id)
    if product is None:
        cache.set_json(key, {"missing": True}, ttl_seconds=30)
        return None

    cache.set_json(key, product, ttl_seconds=300)
    return product

Keep negative TTLs short. If the product is created a second after the miss is cached, users will keep seeing "not found" until that entry expires. Negative caching is there to relieve pressure, nothing more; the source of truth is still where the answer actually lives.

Hot keys

A cache can be perfectly healthy overall and still fall over on a single key. A few that tend to run hot:

global leaderboard
homepage config
latest exchange rate
celebrity profile
tenant with 80 percent of traffic

One Redis key can become the bottleneck even while the cluster as a whole has capacity to spare.

The symptoms are usually clear once you look:

1. High p99 on one endpoint.
2. Redis CPU or network spikes on one shard.
3. One key dominates command stats.
4. Database spikes whenever that key expires.

Mitigations:

1. Replicate the value under several keys and pick one randomly for reads.
2. Cache locally in each app process for a very short TTL.
3. Precompute the value and push updates.
4. Split the value into smaller pieces if reads do not need the whole object.
5. Give the key stronger stampede protection than normal keys.

Local memory caching is often the right answer for small, frequently read data. Even a two-second per-process cache can lift a noticeable amount of pressure off Redis, with the trade that every app instance now holds a slightly different view of the world.

Counters are special

Counters look made for caching: likes, views, followers, notifications. They are also where correctness arguments get hard to settle.

When the number is decorative, an approximate value is enough to satisfy the reader:

1,204 likes
about 1.2K views

When the number controls access or money, the displayed value is driving a real decision, and it needs the source-of-truth figure:

remaining credits
available inventory
account balance

For decorative counters, batch writes:

def record_view(post_id):
    cache.incr(f"views_buffer:post:{post_id}")


def flush_view_counts(limit=1000):
    for key, count in cache.scan_counts("views_buffer:post:*", limit=limit):
        post_id = key.rsplit(":", 1)[1]
        db.increment_post_views(post_id, count)
        cache.delete(key)

This folds many increments into one database write, and in doing so it opens a loss window: if Redis loses the buffer before the flush, those buffered views are simply gone. A lost view leaves the count a little low, which the product can absorb without harm. A payment is held to a different standard, where every cent must be recorded, so it writes straight to the source of truth.

When to skip the cache

Caching is best as a deliberate choice, which means there are good reasons to skip it. Leave the cache out when:

1. The read is already cheap and low volume.
2. The value changes constantly.
3. The key would need too many personalization dimensions.
4. The response contains sensitive data and the cache layer is shared.
5. A cache miss threatens the source of truth while the hit saves little.
6. The team cannot name the invalidation trigger.

A cache you never add is a contract you never have to maintain. Often the real fix lies underneath: an index, a smaller response, pagination, a read model, or a sharper query. Caching is a pressure valve, and it works best laid on top of data that is already modeled correctly.

Metrics that actually help

Hit rate tells one part of the story, not the whole of it. A fuller set:

Hit rate is an average across all reads, and averages can soothe more than they should. A cache at 99 percent hit rate still sends 1 percent of reads to the source of truth, and when that 1 percent lands on a single hot path of expensive queries, those few misses carry the full weight of that path.

What I think

The practical sequence, walked in order, is:

1. Fix the obvious query or data model first.
2. Name the stale promise.
3. Choose the highest safe cache layer.
4. Design the key like a schema.
5. Add TTL with jitter.
6. Invalidate on writes that break the promise.
7. Protect hot misses with single-flight or stale-while-revalidate.
8. Measure misses alongside hits.
9. Keep a repair path back to the source of truth.

It is fine to underengineer a cache, but only when the contract is small and stated plainly:

This value may be 60 seconds stale.
This write invalidates it.
This key includes tenant and viewer.
One caller rebuilds it.
Others get stale data.

That is a design. A cache built on a contract like this one runs on decisions you can point to, and the moment those lines are written down, it stops running on assumptions.

Tutorial checklist

For any cache, it helps to fill this out:

Then ask:

If this cache is empty during peak traffic, what breaks first?

If the answer is "the database," then the cache has quietly become part of capacity planning, and it is better to know that now than during the incident.

Summary

1. Caching is an agreement about acceptable staleness.
2. A cache key is a data model; include every dimension that changes the answer.
3. TTL is a safety fuse; invalidation on write completes the strategy.
4. Cache-aside is a good default, but hot misses need stampede protection.
5. Serve stale data intentionally to keep the source of truth standing under a stampede.
6. Invalidation belongs in the write path.
7. Delete or version cached values on writes, and update in place when every affected key is simple to name.
8. CDN and browser caching depend on precise Cache-Control and Vary headers.
9. Do not cache sensitive or highly personalized data without a precise isolation key.
10. Measure misses, hot keys, stale responses, lock waits, evictions, and source-of-truth load saved.

---

Pop quiz

Interactive quiz

Caching for scale

A randomized review of cache keys, TTLs, invalidation, stampede protection, and stale data promises.

4of 12 questions

Question 1 of 425%

What is the first question to ask before adding a cache?

AWhich cloud provider has the largest Redis instance?BHow stale the value is allowed to be, and for how longCWhether every table can be cached foreverDHow to remove all database reads from the product

Next question