Caching
Where to cache, cache-aside vs read-through vs write-through vs write-back, eviction, invalidation, stampede protection and consistency trade-offs.
What it is
A cache stores the results of expensive operations closer to the consumer so later requests are faster and cheaper. Caches exist at many layers:
| Layer | Examples | Typical content |
|---|---|---|
| Client/browser | HTTP cache, service worker | Static assets, API responses with Cache-Control |
| CDN / edge | CloudFront, Azure Front Door, Cloud CDN, Cloudflare | Images, JS/CSS, cacheable pages and API GETs |
| API gateway | Response caching | Public, identical responses |
| Application (in-process) | Caffeine, Guava | Hot reference data, config |
| Distributed cache | Redis/Valkey, Memcached | Sessions, query results, computed objects |
| Database | Buffer pool, materialized views | Pages, precomputed aggregates |
Caching patterns
| Pattern | Read path | Write path | Notes |
|---|---|---|---|
| Cache-aside (lazy loading) | App reads cache; on miss reads DB and populates cache | App writes DB, then deletes/updates cache | Most common; app controls logic |
| Read-through | Cache library loads from DB on miss | n/a | Simplifies app code |
| Write-through | Reads from cache | Write to cache and DB synchronously | Cache always warm; write latency higher |
| Write-behind (write-back) | Reads from cache | Write to cache; flush to DB asynchronously | Fast writes; data loss risk |
| Refresh-ahead | Proactively refresh before expiry | n/a | Good for predictable hot keys |
sequenceDiagram participant A as App participant C as Redis participant D as Database A->>C: GET product:42 C-->>A: miss A->>D: SELECT ... WHERE id = 42 D-->>A: row A->>C: SET product:42 (TTL 10 min) Note over A,D: On update: write DB, then DEL product:42<br/>(delete rather than set to avoid racing stale writes)
Hard problems
- Invalidation: use TTLs as a safety net plus explicit invalidation on writes, ideally driven by change events (CDC) so all writers are covered.
- Stampede / thundering herd: when a hot key expires, many requests hit the DB at once. Mitigate with request coalescing (single-flight lock), early probabilistic refresh, stale-while-revalidate, and jittered TTLs.
- Hot keys: replicate hot keys across nodes, use local in-process caches in front of Redis.
- Consistency: caches are usually eventually consistent. Do not cache data where staleness breaks correctness (balances used for authorization decisions) unless you version it.
- Penetration: repeated lookups for non-existent keys; cache negative results briefly or use a Bloom filter.
Real-world example
An e-commerce product page caches the rendered product details at the CDN for 60 seconds with stale-while-revalidate, caches price and stock separately in Redis with 5-second TTLs (because they change often), and never caches the checkout total, which is recomputed from the source of truth.
Advantages and disadvantages
| Advantages | Disadvantages |
|---|---|
| Large latency reduction (ms → µs) | Stale data |
| Protects databases from read load | Invalidation bugs are subtle |
| Cheaper per request | Cold-start and failover behaviour must be designed |
| Absorbs spikes | Memory cost; extra component to operate |
Technology options
Redis/Valkey (data structures, persistence options, cluster), Memcached (simple, multi-threaded), managed: Amazon ElastiCache, Azure Managed Redis, Google Memorystore; in-process: Caffeine; CDNs; HTTP caching headers (Cache-Control, ETag, stale-while-revalidate).
[!VERSION] Redis changed licensing in 2024 (RSAL/SSPL), which led to the Linux Foundation fork Valkey; Redis 8 later added AGPLv3 as an option. Cloud providers now offer Valkey-based managed caches. Check licensing for self-managed use.
Architect-level considerations
[!ARCHITECT] For each cached item specify: source of truth, TTL, invalidation trigger, acceptable staleness, behaviour on cache outage (fail open to DB with rate limiting, or degrade), and expected hit ratio.
[!WARNING] If the database cannot survive a full cache outage at peak, the cache is effectively a primary dependency. Plan warm-up, rate-limited fallback and cache replicas accordingly.
Interview questions and answers
Why delete the cache entry on update instead of setting the new value?
Two concurrent writers can set values in the wrong order, leaving stale data until TTL expiry. Deleting forces the next reader to load the latest committed value. Some systems also delay a second delete ("double delete") to cover replica lag.
How do you prevent a cache stampede on a hot key?
Single-flight: only one request recomputes while others wait or get the stale value; serve stale-while-revalidate; refresh early probabilistically before expiry; add TTL jitter so keys do not expire together.
Redis or Memcached?
Redis for rich data structures (sorted sets, streams, hashes), persistence, replication and Lua/functions; Memcached for simple, very fast multi-threaded key-value caching with minimal features.