System design basics (Part 5) — Caching

A cache is a piece of hardware or software that stores data, and is typically meant to retrieve that data faster than it otherwise would.

Caches are often used to store responses to network requests, as well as results of computationally-long operations.

However, do note that data in a cache can become stale if the main source of truth for that data (i.e. the main database behind the cache) gets updated and the cache doesn’t.

Now, how does caching help?

1. It can improve the time performance of algorithms, as data stored through caching can speed up or reduce latency in the system
2. Hence, it can be used for:

a. Processes with a larger number of network requests

b. Computationally complex requests

c. Capturing information to be used on a repeated basis (such as user login information); this static content can be cached rather than always obtaining it via a network call

There are a couple of types of caches, broadly speaking.

1. A write through cache, will write the data both in cache as well as in the database (which, is the source of truth); this ensures both are in sync
2. A write back cache, will write the data in cache only, and will asynchronously sync with the database; in this case, the cache can go stale

Now, stale caches can be an issue, but only for data that NEEDS to be accurate at all times; instances where the cache can be updated with the next network call can then be overwritten and then sync with the database

Caching can, of course, be applied at multiple levels:

1. At client level (data stored from server)
2. At server level (data stored from database)
3. At database level

Taking this concept to a real world service, a content delivery network, or CDN is a third-party service that acts like a cache for your servers. Sometimes, web applications can be slow for users in a particular region if your servers are located only in another region.

(Example — you are trying to stream video in one country, but all servers of the video streaming platform are located on the other side of the world)

A CDN has servers all around the world, meaning that the latency to a CDN’s servers will almost always be far better than the latency to your servers. A CDN’s servers are often referred to as PoPs (points of presence).

A couple of popular CDNs are Cloudflare and Google Cloud CDN.