If I could teach a junior engineer exactly one system design skill, it would be caching. Nothing else gives you as much speed for as little effort — a well-placed cache can turn a database that's melting under load into one that's barely warm. It's also the place people make the most subtle mistakes, because a cache is a second copy of your data, and two copies of anything eventually disagree.
This is part four of my system design fundamentals series. Let's make things fast, then talk about the ways caching quietly bites back.
Why caching works
Remember the latency numbers from earlier in this series: memory is dramatically faster than disk, which is dramatically faster than the network. Caching is just the disciplined exploitation of that gap — you keep a copy of expensive-to-fetch data somewhere cheap-to-reach, so the next request skips the slow path entirely.
The best cache targets are data that's read far more often than it's written, and expensive to produce: the result of a heavy query, a rendered page fragment, a user's profile, a computed feed. If something is written as often as it's read, or trivial to compute, caching it buys you little and adds a consistency headache for nothing.
Caching happens at many levels
It helps to picture caching as a series of checkpoints between the user and your database, each one a chance to answer before the request travels further.
- Client-side — the browser or app caches responses and assets locally. The fastest possible cache, because the request never leaves the device.
- CDN — a network of edge servers caches static content close to users geographically (more on this below).
- Application / in-memory — a dedicated cache like Redis or Memcached sits beside your app servers, holding hot data so you skip the database.
- Database — databases keep their own internal caches of frequently accessed pages and query results.
Each layer you add absorbs load before it reaches the next. A request that's answered at the CDN never touches your servers at all — which is exactly the point.
The patterns: how data gets into and out of the cache
This is where the real decisions live. There are a few standard strategies, and choosing the wrong one causes real bugs.
Cache-aside (lazy loading) is the most common. Your application checks the cache first; on a miss, it reads from the database, writes the result into the cache, and returns it. Simple and resilient — if the cache goes down, everything still works, just slower. The downsides are that the first request for any item is always slow (a cache miss), and if data changes in the database, the cache can serve stale values until the entry expires or you explicitly evict it.
Write-through writes to the cache and the database together on every write. The cache is always fresh, which is great for read-heavy data you can't afford to serve stale. The cost is slower writes (two writes every time) and the cache filling with data that may never be read.
Write-behind (write-back) writes to the cache immediately and flushes to the database asynchronously a moment later. Writes feel blazing fast because the slow database write happens off the critical path. The danger is real: if the cache dies before the flush, you lose data. It's powerful and risky, so reserve it for cases where a little data-loss risk is acceptable in exchange for write speed.
There's no default-correct answer. Cache-aside for most read paths, write-through when staleness is unacceptable, write-behind only when you truly need write throughput and can tolerate the risk.
Eviction and expiry: caches are finite
A cache has limited memory, so when it fills up, something has to go. The eviction policy decides what.
- LRU (Least Recently Used) — evict whatever hasn't been touched in the longest time. A sensible default, because recently used things tend to get used again.
- LFU (Least Frequently Used) — evict whatever is accessed least often. Better when popularity is stable over time.
- FIFO — evict the oldest entry regardless of use. Simple but usually worse than LRU in practice.
Separately, you set a TTL (time to live) — how long an entry is allowed to live before it's considered stale and refetched. TTL is your main lever against serving outdated data: short TTLs keep things fresh but shift load back to the database; long TTLs save load but risk staleness. Tuning it is a per-dataset judgment call, and "how wrong can this be for how long?" is the question to ask.
Redis vs Memcached
The two names you'll hear constantly. Memcached is a lean, fast, in-memory key-value cache — dead simple, great at exactly one thing: caching strings/objects by key. Redis does that too but adds rich data structures (lists, sets, sorted sets, hashes), optional persistence to disk, replication, and pub/sub. In practice Redis has become the default because that extra capability is genuinely useful — sorted sets alone power leaderboards, rate limiters, and queues. I reach for Memcached only when I want the absolute simplest cache and nothing more; otherwise Redis.
Content Delivery Networks (CDNs)
A CDN is caching applied to geography. It's a network of servers spread around the world that cache your static content — images, CSS, JavaScript, videos — and serve each user from the location nearest to them. Instead of every request crossing an ocean to your origin server, it's answered a few milliseconds away.
The benefits stack up: dramatically lower latency for users far from your servers, a huge amount of traffic offloaded from your origin, and a side effect of DDoS resilience, since the CDN's massive distributed capacity absorbs floods that would flatten a single origin.
The hard part of CDNs is invalidation — getting the edge to stop serving an old version after you deploy a change. Two common approaches: set sensible cache-control headers so content expires on a schedule, and use versioned URLs (like app.a1b2c3.js) so a new version is simply a new URL the CDN has never cached. Versioning sidesteps the whole staleness problem elegantly, which is why it's the standard trick for assets.
The one rule to remember
Caching is the highest-leverage performance tool you have, and every cache is a promise that two copies of your data will agree. The bugs almost always come from that second half — stale reads, lost writes, invalidation you forgot. So cache aggressively, but always answer three questions up front: what am I caching, how does it get updated, and how wrong can it be before that becomes a problem? Get those right and caching feels like magic. Get them wrong and you'll spend a weekend debugging why a user sees data that no longer exists.
Next in the series, we go to the heart of most systems: databases — SQL vs NoSQL, and how sharding and replication let your data layer scale past a single machine.
