Caching

One-liner: Keep hot data in a faster, closer tier to cut latency and shed load from the source; the hard parts are invalidation and stampedes.

Say this#

SAY THIS

Cache-aside (lazy loading) — app checks cache, on miss reads DB and populates cache. Most common.
Read-through — cache itself loads from DB on miss; app only talks to cache.
Write-through — write goes to cache and DB synchronously; consistent, slower writes.
Write-back / write-behind — write to cache, flush to DB asynchronously; fast, risk of loss.
Write-around — write to DB only; cache fills on next read; avoids polluting cache with cold writes.
Eviction policies — LRU (recency), LFU (frequency), FIFO, random; TTL expiry is separate from eviction.
Cache stampede / thundering herd / dog-piling — many requests miss at once and hammer the DB.
Request coalescing / single-flight — only one request recomputes; others wait.
Stale-while-revalidate — serve stale, refresh in background.
Probabilistic early expiration — refresh slightly before TTL with random jitter.
Hit ratio — hits / (hits + misses); the number to quote.
Negative caching — cache "not found" to stop repeated misses.
Hot key — one key gets disproportionate traffic; replicate it or add local cache.

How it works#

Cache layers, outermost first:

TEXT

browser cache (Cache-Control, ETag) -> CDN (CloudFront, Cloudflare) -> API gateway / reverse proxy (Nginx, Varnish)  -> application in-process (LRU map, Caffeine, Guava) -> distributed cache (Redis, Memcached) -> DB buffer poolbrowser cache (Cache-Control, ETag) -> CDN (CloudFront, Cloudflare) -> API gateway / reverse proxy (Nginx, Varnish)
  -> application in-process (LRU map, Caffeine, Guava) -> distributed cache (Redis, Memcached) -> DB buffer pool

Cache-aside flow:

GET key from Redis. Hit → return.
Miss → read DB → SET key with TTL → return.
On write: update DB, then DELETE the cache key (don't SET; avoids race with stale read). Next read repopulates.

Stampede fixes: per-key lock/single-flight, stale-while-revalidate, jittered TTLs, request coalescing at the proxy, pre-warming before deploys/traffic spikes.

Trade-offs#

Choice	Pro	Con
Cache-aside	Simple, only hot data cached, cache failure is survivable	First read is a miss; stale window until TTL/invalidation
Write-through	Cache always fresh	Write latency doubles; caches cold data
Write-back	Fastest writes, absorbs bursts	Data loss if cache dies before flush; complex
Redis	Rich types, persistence, pub/sub, Lua, single-threaded simplicity	Single-threaded per core; memory cost
Memcached	Multithreaded, simple, very fast for plain KV	No persistence, no data structures
In-process cache	Nanosecond access, no network	Per-instance inconsistency; memory duplicated

Numbers#

In-process lookup ~100 ns; Redis/Memcached over LAN ~0.2-1 ms; DB query several ms to tens of ms.
Redis single node: ~100k ops/s order of magnitude (more with pipelining); Memcached scales with cores.
A 90% hit ratio cuts DB load 10x; 99% cuts it 100x. Quote the ratio.
TTLs: seconds for volatile data, minutes-hours for profile-like data; add ±10% jitter.

Say this#

How it works#

Trade-offs#

Numbers#

In the interview#