Caching

Where to cache, cache-aside vs read-through vs write-through vs write-back, eviction, invalidation, stampede protection and consistency trade-offs.

What it is

A cache stores the results of expensive operations closer to the consumer so later requests are faster and cheaper. Caches exist at many layers:

Layer Examples Typical content
Client/browser HTTP cache, service worker Static assets, API responses with Cache-Control
CDN / edge CloudFront, Azure Front Door, Cloud CDN, Cloudflare Images, JS/CSS, cacheable pages and API GETs
API gateway Response caching Public, identical responses
Application (in-process) Caffeine, Guava Hot reference data, config
Distributed cache Redis/Valkey, Memcached Sessions, query results, computed objects
Database Buffer pool, materialized views Pages, precomputed aggregates

Caching patterns

Pattern Read path Write path Notes
Cache-aside (lazy loading) App reads cache; on miss reads DB and populates cache App writes DB, then deletes/updates cache Most common; app controls logic
Read-through Cache library loads from DB on miss n/a Simplifies app code
Write-through Reads from cache Write to cache and DB synchronously Cache always warm; write latency higher
Write-behind (write-back) Reads from cache Write to cache; flush to DB asynchronously Fast writes; data loss risk
Refresh-ahead Proactively refresh before expiry n/a Good for predictable hot keys
sequenceDiagram
  participant A as App
  participant C as Redis
  participant D as Database
  A->>C: GET product:42
  C-->>A: miss
  A->>D: SELECT ... WHERE id = 42
  D-->>A: row
  A->>C: SET product:42 (TTL 10 min)
  Note over A,D: On update: write DB, then DEL product:42<br/>(delete rather than set to avoid racing stale writes)

Hard problems

  • Invalidation: use TTLs as a safety net plus explicit invalidation on writes, ideally driven by change events (CDC) so all writers are covered.
  • Stampede / thundering herd: when a hot key expires, many requests hit the DB at once. Mitigate with request coalescing (single-flight lock), early probabilistic refresh, stale-while-revalidate, and jittered TTLs.
  • Hot keys: replicate hot keys across nodes, use local in-process caches in front of Redis.
  • Consistency: caches are usually eventually consistent. Do not cache data where staleness breaks correctness (balances used for authorization decisions) unless you version it.
  • Penetration: repeated lookups for non-existent keys; cache negative results briefly or use a Bloom filter.

Real-world example

An e-commerce product page caches the rendered product details at the CDN for 60 seconds with stale-while-revalidate, caches price and stock separately in Redis with 5-second TTLs (because they change often), and never caches the checkout total, which is recomputed from the source of truth.

Advantages and disadvantages

Advantages Disadvantages
Large latency reduction (ms → µs) Stale data
Protects databases from read load Invalidation bugs are subtle
Cheaper per request Cold-start and failover behaviour must be designed
Absorbs spikes Memory cost; extra component to operate

Technology options

Redis/Valkey (data structures, persistence options, cluster), Memcached (simple, multi-threaded), managed: Amazon ElastiCache, Azure Managed Redis, Google Memorystore; in-process: Caffeine; CDNs; HTTP caching headers (Cache-Control, ETag, stale-while-revalidate).

[!VERSION] Redis changed licensing in 2024 (RSAL/SSPL), which led to the Linux Foundation fork Valkey; Redis 8 later added AGPLv3 as an option. Cloud providers now offer Valkey-based managed caches. Check licensing for self-managed use.

Architect-level considerations

[!ARCHITECT] For each cached item specify: source of truth, TTL, invalidation trigger, acceptable staleness, behaviour on cache outage (fail open to DB with rate limiting, or degrade), and expected hit ratio.

[!WARNING] If the database cannot survive a full cache outage at peak, the cache is effectively a primary dependency. Plan warm-up, rate-limited fallback and cache replicas accordingly.

Interview questions and answers

Why delete the cache entry on update instead of setting the new value?

Two concurrent writers can set values in the wrong order, leaving stale data until TTL expiry. Deleting forces the next reader to load the latest committed value. Some systems also delay a second delete ("double delete") to cover replica lag.

How do you prevent a cache stampede on a hot key?

Single-flight: only one request recomputes while others wait or get the stale value; serve stale-while-revalidate; refresh early probabilistically before expiry; add TTL jitter so keys do not expire together.

Redis or Memcached?

Redis for rich data structures (sorted sets, streams, hashes), persistence, replication and Lua/functions; Memcached for simple, very fast multi-threaded key-value caching with minimal features.