Memcache is a technology that helps web apps and mobile app backends in two main ways: performance and scalability. We’ll discuss each of these in detail below, but first we’ll introduce memcache briefly.
What is memcache?
Memcache is an in-memory, distributed cache. The primary API for interacting with it are
SET(key, value) and
GET(key) operations. Memcache is essentially a hashmap (or dictionary) that is spread across multiple servers, where operations are still performed in constant time.
The most common usage of memcache is to cache expensive database queries and HTML renders such that these expensive operations don’t need to happen over and over again.
Memcache reduces page load time, giving your user a faster, better experience. Remember, speed is a feature. Without memcache, each page load will need to perform database queries and HTML rendering over and over again. Often these operations take several hundred milliseconds, but sometimes they can take seconds depending on complexity and server load. Memcache queries return in milliseconds, which is usually 100x faster than a database query or a complex render (again, depending on load and complexity). This allows any database queries or renders that are common to some of your users to be shared. Even operations unique to a user can be useful to cache if the same result will be displayed to them on every page.
If you’re starting with an empty cache, the first page load will execute in the same time as it would without memcache. However, after expensive operations have been cached during the first page load, that same page load will be drastically faster each subsequent time until your key is either expired or evicted from the cache. More on eviction and expiration later.
An app’s scalability is a function of throughput — the more throughput your app has, the better it will scale. Memcache gives your app more throughput because of two related reasons. The first is simple, by avoiding slower database queries, page renders or web API calls your app can handle a request faster and so execute more per second. The second component of scalability is by hitting the cache you avoid touch the database and disk. This is generally highly relied upon to scale out the database. Without memcache, many apps would be hitting their database too often, making it a bottleneck for their app. While they could try to directly scale the database with say master / slave replication, these are complicated technologies to setup and use and aren’t as effective as simply putting memcache in between your app and the database for a lot of use cases.
How is memcache so fast?
Memcache is so fast for two reasons: the cache is entirely in-memory; and operations are performed in constant time. Memcache avoids writing any data to disk, which results in faster operations and no fault tolerance. If the server running memcache is restarted, the keys on that server will be lost, which is fine for the use case of memcache, as a temporal caching layer and not a definitive data store. The use of constant time operations is simply a deployment of all those good old tricks and tips for a computer science algorithms & data structures class. O(1) is a nice property to have.
Eviction, Expiration, and Memory Limits
Besides getting and setting keys, eviction and expiration are major components of memcache, too. When a key is SET in the cache, an expiration is set along with the key. Most memcache clients have a default expiration which can be optionally overwritten on a per-key basis.
Memcache doesn’t evict keys when their expiration time expires, as doing so isn’t possible while still guaranteeing O(1) operation times. Instead expiration is more of a way to say how long until a key should be considered stale. When a GET is performed, Memcache checks if the key’s expiration time is still valid before returning it.
A key is evicted from the cache when the total cache size has reached the limit. Most memcache implementations offer a least recently used (LRU) eviction policy, which evicts the key that was least recently used when a new key is added to the cache and the cache has reached its limit.
In general, the higher your cache limit, the fewer evictions you’ll have, and the higher your hit-rate will be, ultimately resulting in better performance and scalability.
How can I check query and render times?
Most web frameworks provide a mechanism to log page load operations. For example, in Rails, the development log looks something like this:
... Application Load (16.3ms) SELECT `customers`.* FROM `applications` Rendered customers/_list.html.erb (185.8ms) ...
In the log above we see that the
SELECT statement ran in 16.3ms, and the render of
customers/_list.html.erb ran in 185.8ms.
In addition to logs, tools like New Relic analyze performance in real time and give you reports about page load times.
You should use memcache when you want faster page loads and/or more scalability. In fact, if you expect or are hoping that your website or mobile app will need to scale at some point then it is often a good development practice to use memcache from the start. While you won’t have an actual need for it in the beginning, designing your app to use memcache initially is far easier than plugging it in later.