Django persistent memcached connections

Recently we diagnosed an issue for several of our customers who were using the python Django web-framework. They were all experiencing performance issues and a reasonable amount of broken connections to our servers. The issue was that by default Django uses a new connection to memcached for each request. This is terrible for performance in general, paying the cost of an extra round-trip to setup the TCP connection, but is far worse with cloud-services like MemCachier that have an security layer and require new connections to be authenticated. A new connection to a MemCachier server takes at least 3 round-trips, which increases the time to execute a single get request by 4x.

It turned out this situation is the result of a very old and long-standing bug in Django (#11331). This bug arose due to a incorrect fix for another bug that Django had, one where it could continually create new connections to the memcached server without ever closing any of them until it eventually consumed all the available file descriptors at the server. Unfortunately this fix for this file descriptor leak was to simply close the memcached connection after all requests.

Thankfully we can fix this by monkey-patching Django and disabling the close method of the Memcached class. The best place to do this is in your Django wsgi.py file, inserting the code below before you create your application:

That is, the full file may look something like follows:

We’ve advised several customers to use this fix and they’ve had great results! We will soon be adding it to our docs as a recommended practice for Django.

6 thoughts on “Django persistent memcached connections

  1. I am facing similar issue with django. I just have a little concern, won’t this monkey patch re-originate the earlier problem which Django had fixed??
    “one where it could continually create new connections to the memcached server without ever closing any of them”
    Or this will give me a singleton kind of object for all memcached requests?

  2. Hello, my good man! I’m experiencing an extended problem of this — pylibmc resets its failover state on calling disconnect_all, so if a server goes down, Django will keep resetting the failure state of that server. Did you ever get around to introducing this to Django? I opened a pull request with the monkey-patch you supplied above added in.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s