Deploy Django and Memcache on Render: A How-To Guide

Paddy ~ May 31, 2023

Tags: Render, Django, Tutorial

Do you want to deploy a Django application on Render and boost its performance with Memcache? In this tutorial, you’ll create a simple Django task list app, deploy it to Render, and finally use Memcache to speed it up.

Memcache serves as an in-memory data storage system, a tool that significantly enhances the efficiency and scalability of web applications. Consider integrating Memcache if your pages exhibit sluggish load times or your application faces scaling challenges. Regardless of your app’s size, Memcache has the potential to quicken page loads and help future-proof your app.

Outline

• Initialize a Django project
• Create a Django task list app
• Configure your app for Render
• Deploy Django to Render
• Set up caching in Django
• Implement caching strategies in Django

Prerequisites

• Familiarity with Python (and ideally Django).
• A Render account.
• A GitHub account and Git installed on your local machine. To deploy to Render, you must push code to a remote code repository. Render supports GitHub and GitLab.
• Python (version >= 3.8) installed on your computer. I’ll use 3.11.1.

Initialize a Django project

The following commands will create an isolated Python virtual environment and bootstrap a starter Django project.

Create a new folder for the project and change into that directory:

mkdir django-memcache && cd django-memcache

Make sure your terminal window is using at least Python 3.8. 3.8 is the minimum required version for Django 4.2, which you’ll use in this tutorial. Then, create a new Python virtual environment.

python -m venv venv

When you run this command, Python creates a new directory called venv. venv allows you to install additional packages and dependencies into this virtual environment without affecting your system’s global Python installation. -m venv tells Python to use the venv module to create a new virtual environment. The final venv is the directory’s name where the virtual environment will be created. You can choose any name, but venv is a common convention.

Next, activate the Python virtual environment:

source venv/bin/activate

After running that command, you should see the name of the virtual environment displayed in your terminal prompt, e.g., (venv) ➜ django-memcache.

Install the latest version of Django (4.2 at the time of writing) using the pip package installer:

(venv) $ python -m pip install Django

Next, create a new Django project:

(venv) $ django-admin startproject myproject .

This command creates a new Django project named myproject in the current directory (.). django-admin is Django’s command-line utility for administrative tasks.

Next, start the Django development server:

(venv) $ python manage.py runserver

manage.py does the same thing as django-admin but also sets the DJANGO_SETTINGS_MODULE environment variable so that it points to your project’s settings.py file. That tells Django which settings to use for your project.

# Output
Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).

You have 18 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): admin, auth, contenttypes, sessions.
Run 'python manage.py migrate' to apply them.
May 30, 2023 - 12:29:57
Django version 4.2.1, using settings 'myproject.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

Visit http://127.0.0.1:8000/ in your browser. You’ll see a Congratulations! page, with a rocket taking off.

Create a Django task list app

The Django application we are building is a task manager. In addition to displaying a list of tasks, it will have actions to add new tasks and to remove them. To accomplish this, we need to:

1. Create a Django app.
2. Create a Task model.
3. Run database migrations locally.
4. Add the route, view, and controller logic for adding, removing, viewing a task, and viewing all tasks.

Create a Django app

Django has the concept of apps, and we must create one to add functionality. Create an app named myapp:

(venv) $ python manage.py startapp myapp

Add myapp to the list of installed apps in myproject/settings.py:

# myproject/settings.py
# ...

INSTALLED_APPS = [
    'myapp.apps.MyappConfig',
    # ...
]

# ...

See the Configuring applications Django documentation for a detailed explanation of this configuration.

Create a Task model

Create a Task model in myapp/models.py:

# myapp/models.py
from django.db import models

class Task(models.Model):
    name = models.TextField()
    notes = models.TextField()

A Task has a name, and can have some notes, which we’ll make optional.

Next, create a migration for the myapp app:

(venv) $ python manage.py makemigrations myapp

# Output
Migrations for 'myapp':
  myapp/migrations/0001_initial.py
    - Create model Task

Run database migrations locally

Run database migrations locally to create the myapp_task table, along with all other default Django tables:

(venv) $ python manage.py migrate

# Output
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, myapp, sessions
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying auth.0012_alter_user_first_name_max_length... OK
  Applying myapp.0001_initial... OK
  Applying sessions.0001_initial... OK

With the local database setup complete, you’re ready to add some functionality to your app.

Add views

Next, you’ll create the following four views:

• index: display all tasks.
• detail: view a single task.
• add: add a task.
• remove: remove a task.

In myapp/views.py, replace the contents of the file with the following:

# myapp/views.py
from django.template.context_processors import csrf
from django.shortcuts import render, redirect
from myapp.models import Task

def index(request):
    tasks = Task.objects.order_by('id')
    context = {'tasks': tasks}
    return render(request, 'index.html', context)

def detail(request, task_id):
    task = Task.objects.get(id=task_id)
    context = {'task': task}
    return render(request, 'detail.html', context)

def add(request):
    if 'name' in request.POST:
        task = Task(name=request.POST['name'], notes=request.POST['notes'])
        task.save()
        return redirect('/')
    return render(request, 'add.html')

def remove(request):
    task = Task.objects.get(id=request.POST['id'])
    if task:
        task.delete()
    return redirect('/')

• The index view gets all tasks from the database and passes them as the template’s context.
• The detail view gets a task from the database and passes it as the template’s context.
• The add view checks if a POST value named name is in the request. If so, a task is created and saved to the database, after which a redirect to the index view happens. Otherwise, the add view is rendered.
• The remove view gets a task from the database by ID. If the item exists, it is deleted. Then, a redirect to the index view happens.

The render() function takes the request object as its first argument, a template name as its second argument, and a dictionary as its optional third argument. You will create the corresponding templates next.

Create the index.html template

Now, create the template to display all tasks when the index view is requested. Create the file myapp/templates/index.html and add the following markup:

<!-- myapp/templates/index.html -->
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <title>MemCachier Django tutorial</title>
    <meta name="viewport" content="width=device-width, initial-scale=1" />
  </head>
  <body>
    <a href="/add">Add a Task</a>

    <h1>Tasks</h1>
    {% if tasks %}
    <ul>
      {% for task in tasks %}
      <li>
        <a href="/detail/{{ task.id }}">{{ task.name }}</a>
      </li>
      {% endfor %}
    </ul>
    {% else %}
      <a href="/add">Add your first task</a>
    {% endif %}
  </body>
</html>

The template contains a link to the add task view and lists existing tasks. A link to Add your first task is displayed if no tasks exist.

Create the detail.html template

Next, add a template for the detail view. Create the file myapp/templates/detail.html and add the following markup:

<!-- myapp/templates/detail.html -->
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <title>MemCachier Django tutorial | Task View</title>
    <meta name="viewport" content="width=device-width, initial-scale=1" />
  </head>
  <body>
    <a href="/">Back to Tasks</a>
    <h1>{{ task.name }}</h1>
    <p>{{ task.notes }}</p>
    <form action="/remove" method="POST">
      {% csrf_token %}
      <input type="hidden" name="id" value="{{ task.id }}" />
      <button>Delete Task</button>
    </form>
  </body>
</html>

This template displays the name and notes for a task and a form with a delete button for removing a task.

Create the add.html template

Finally, create the add view. Create the file myapp/templates/add.html and add the following markup:

<!-- myapp/templates/add.html -->
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <title>MemCachier Django tutorial | Add a Task</title>
    <meta name="viewport" content="width=device-width, initial-scale=1" />
  </head>
  <body>
    <a href="/">Back to Tasks</a>
    <h1>New Task</h1>
    <form action="/add" method="POST">
      {% csrf_token %}
      <div>
        <label>
          Task Name:
          <input
            type="text"
            name="name"
            placeholder="e.g. Wash the car"
            required
          />
        </label>
      </div>
      <div>
        <label>
          Task Notes:
          <textarea name="notes"></textarea>
        </label>
      </div>
      <button>Add Task</button>
    </form>
  </body>
</html>

This template contains a task creation form, with input fields for task name and optional notes.

Route views to URLs

To call these views, we need to map them to URLs by adding the following in myproject/urls.py:

# myproject/urls.py
# ...
from myapp import views

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', views.index),
    path('detail/<int:task_id>', views.detail),
    path('add', views.add),
    path('remove', views.remove),
]

Our task list is now functional. With the Django development server running, python manage.py runserver, visit http://127.0.0.1:8000/ in your browser and try it out. You should be able to add and delete tasks, to view all tasks, and to view a single task.

Configure your app for Render

To prepare your Django app to be deployed to Render, you’ll do the following:

• Configure Django to use PostgreSQL
• Create a Render build script
• Update Django’s settings to allow it to serve your Render URL
• Define your Render infrastructure as code
• Create a remote code repository for your app

Configure Django to use PostgreSQL

Render databases provide connection strings. You’ll use the DJ-Database-URL package to enable Django to use connection strings.

(venv) $ pip install dj-database-url

You also need to install the psycopg2 package. psycopg2 is a PostgreSQL adaptor for Python. It’s needed for the Django ORM to communicate with PostgreSQL.

(venv) $ pip install psycopg2-binary

In myproject/settings.py import dj_database_url and update the DATABASES setting:

# myproject/settings.py
# ...
import dj_database_url

# ...

def get_db():
    try:
        return {
            # Looks for the environment variable DATABASE_URL
            'default': dj_database_url.config(
                conn_max_age=600,
                conn_health_checks=True,
            ),
        }
    except:
        return {
            'default': {
                'ENGINE': 'django.db.backends.sqlite3',
                'NAME': BASE_DIR / 'db.sqlite3',
            }
        }

DATABASES = get_db()

# ...

The get_db() function first attempts to read the environment variable DATABASE_URL using the dj_database_url.config method. If there’s an exception, such as if the DATABASE_URL variable is not found, an SQLite3 database will instead be used. This means that SQLite3 will continue to work locally.

The DJ-Database-URL README explains the dj_database_url.config settings as follows:

conn_max_age tells Django to persist database connections between requests, up to the given lifetime in seconds. If you do not provide a value, it will follow Django’s default of 0. Setting it is recommended for performance.

conn_health_checks tells Django to check a persisted connection still works at the start of each request. If you do not provide a value, it will follow Django’s default of False. Enabling it is recommended if you set a non-zero conn_max_age.

Create a Render build script

You’ll next create a build script for Render to install Python dependencies and run database migrations. Create a file build.sh in your project root directory and add the following:

#!/usr/bin/env bash
# exit on error
set -o errexit

pip install -r requirements.txt

python manage.py migrate

Make the script executable:

(venv) $ chmod a+x build.sh

The chmod command changes the file’s permissions, and a+x means “add execute permission for all users”.

In the step, Define your Render infrastructure as code, you’ll configure Render to run this script on every git push.

Update Django’s settings to allow it to serve your Render URL

For Django to serve a particular host or domain name in production, it must be defined in the ALLOWED_HOSTS setting. This is a security measure. Render generates a dynamic URL for your environment, e.g., https://myapp.onrender.com. You’ll specify that with the RENDER_EXTERNAL_HOSTNAME environment variable:

In myproject/settings.py add the following under the ALLOWED_HOSTS setting:

# myproject/settings.py
# ...
import os

# ...

ALLOWED_HOSTS = []

RENDER_EXTERNAL_HOSTNAME = os.environ.get('RENDER_EXTERNAL_HOSTNAME')
if RENDER_EXTERNAL_HOSTNAME: ALLOWED_HOSTS.append(RENDER_EXTERNAL_HOSTNAME)
# ...

Define your Render infrastructure as code

Create a render.yaml Blueprint spec file in your project root directory. You’ll define a Django Web Service and PostgreSQL Database in the spec.

# render.yaml
databases:
  - name: myproject
    plan: free
    databaseName: myproject
    user: myproject

services:
  - type: web
    name: myproject
    plan: free
    env: python
    buildCommand: "./build.sh"
    startCommand: "gunicorn myproject.wsgi:application"
    envVars:
      - key: PYTHON_VERSION
        value: 3.11.1
      - key: DATABASE_URL
        fromDatabase:
          name: myproject
          property: connectionString
      - key: SECRET_KEY
        generateValue: true
      - key: WEB_CONCURRENCY
        value: 4

The default Python version on Render is 3.7 at the time of writing. Django 4.2 requires at least Python 3.8, so you set the PYTHON_VERSION to change your Render service Python version.

The buildCommand property runs the build.sh script you created in earlier.

Note you’re specifying Render’s Free plan, which is sufficient for this tutorial. The default plan is Starter. See the Render Blueprint spec documentation for an explanation of all properties.

The command gunicorn myproject.wsgi:application starts your Django application using Gunicorn, a Python WSGI HTTP Server. Install the gunicorn package:

(venv) $ pip install gunicorn

Next, use pip freeze to write your dependencies to a file named requirements.txt:

(venv) $ pip freeze > requirements.txt

This file is required to tell Render what Python dependencies to install.

Create a remote code repository for your app

To deploy to Render, you need a remote code repository. In this step, you’ll initialize a Git repository and push it to GitHub.

Start by initializing a Git repo:

git init

Create a .gitignore file and add the following:

# .gitignore
venv
*.pyc
db.sqlite3

# macOS file
.DS_Store

.DS_Store is a macOS-specific file that doesn’t need to be present on other operating systems.

Add your changes:

git add .

Commit those changes:

git commit -m 'Create Django app for Render'

You’ll see output similar to the following:

# output
[main (root-commit) ebabd95] Create Django app for Render
 21 files changed, 408 insertions(+)
 create mode 100644 .gitignore
 create mode 100755 build.sh
 create mode 100755 manage.py
 create mode 100644 myapp/__init__.py
 create mode 100644 myapp/admin.py
 create mode 100644 myapp/apps.py
 create mode 100644 myapp/migrations/0001_initial.py
 create mode 100644 myapp/migrations/__init__.py
 create mode 100644 myapp/models.py
 create mode 100644 myapp/templates/add.html
 create mode 100644 myapp/templates/detail.html
 create mode 100644 myapp/templates/index.html
 create mode 100644 myapp/tests.py
 create mode 100644 myapp/views.py
 create mode 100644 myproject/__init__.py
 create mode 100644 myproject/asgi.py
 create mode 100644 myproject/settings.py
 create mode 100644 myproject/urls.py
 create mode 100644 myproject/wsgi.py
 create mode 100644 render.yaml
 create mode 100644 requirements.txt

Now your code is committed, you’ll push it to GitHub.

In your browser, log in to GitHub and create an empty repository called render-django. The repo can be public or private.

Back in the terminal, add your GitHub repo as a remote origin, replacing your_username with your actual GitHub username:

git remote add origin https://github.com/your_username/render-django.git

That command tells Git where to push your code to.

Rename the default branch:

git branch -M main

And push your code to GitHub:

git push -u origin main

You’ll see output similar to the following:

# output
Enumerating objects: 25, done.
Counting objects: 100% (25/25), done.
Delta compression using up to 8 threads
Compressing objects: 100% (23/23), done.
Writing objects: 100% (25/25), 6.00 KiB | 3.00 MiB/s, done.
Total 25 (delta 1), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (1/1), done.
To https://github.com/your_username/render-django.git
 * [new branch]      main -> main
branch 'main' set up to track 'origin/main'.

Your app’s code is now on GitHub, ready to be deployed to Render.

Deploy Django to Render

Now that your code is on GitHub, deploying to Render is as simple as creating a new Blueprint Instance and connecting your GitHub repository.

Log in to Render and go to your Dashboard. Click New +, then click Blueprint.

Connect your GitHub account if you still need to do so. Then, find your repo in the Connect a repository section of the Create a new Blueprint Instance page and click Connect.

After clicking Connect you’ll be taken to a settings page. Choose a Blueprint Name. I will use render-django. And click Apply to begin deploying.

When the deployment is live, go to your Render Dashboard, click on your Web Service, and finally click on its URL under the Web Service name to open it in your browser. The task list app should work as it does locally.

Next, you’ll implement caching.

Set up caching in Django

To set up caching in Django you will do the following steps:

• Create a Memcached cache
• Configure Django to use your cache

Create a Memcached cache

In this tutorial, you’ll create a free Memcached-compatible cache with MemCachier. For another option to use Memcached on Render, read our blog post to learn how to run Memcached as a Render Private Service.

MemCachier is a fully managed caching service that simplifies setting up and using Memcached in your web applications. Built for seamless integration with cloud platforms like Render, MemCachier offers developers a streamlined and hassle-free way to implement high-performance caching in their applications without worrying about the complexities of managing a cluster of Memcached servers.

To begin, create a new MemCachier cache. Choose Render as the provider. Choose the same region as your Render Web Service. Choose the Free plan. Finally, click CREATE CACHE.

After creating your cache, you’ll find its configuration settings (Username, Password, Servers) on the CACHES dashboard.

On your Render dashboard, select your Web Service, then select the Environment tab. Then, add your MemCachier Servers, Username, and Password config values, naming the environment variables MEMCACHIER_SERVERS, MEMCACHIER_USERNAME, and MEMCACHIER_PASSWORD, respectively.

Click Save Changes. Next, you’ll configure Django to use your cache.

Configure Django to use your cache

To begin, install the django-bmemcached Django cache backend. That package uses the bmemcached module, which supports the Memcached binary protocol with SASL authentication, allowing you to connect to your cache with a username and password.

(venv) $ pip install django-bmemcached

And update your dependencies list:

(venv) $ pip freeze > requirements.txt

Configure Django to use your MemCachier cache by adding the following to the end of myproject/settings.py:

# myproject/settings.py
# ...
def get_cache():
    try:
        servers = os.environ['MEMCACHIER_SERVERS']
        username = os.environ['MEMCACHIER_USERNAME']
        password = os.environ['MEMCACHIER_PASSWORD']

        return {
            'default': {
                'BACKEND': 'django_bmemcached.memcached.BMemcached',
                # TIMEOUT is not the connection timeout! It's the default expiration
                # timeout that should be applied to keys! Setting it to `None`
                # disables expiration.
                'TIMEOUT': None,
                'LOCATION': servers,
                'OPTIONS': {
                    'username': username,
                    'password': password,
                }
            }
        }
    except:
        return {
            'default': {
                'BACKEND': 'django.core.cache.backends.locmem.LocMemCache'
            }
        }

CACHES = get_cache()

This code configures the cache for both development and production. If the MEMCACHIER_* environment variables exist, the cache will be set up with django_bmemcached, connecting to MemCachier. Whereas, if the MEMCACHIER_* environment variables don’t exist—hence development mode—Django’s simple in-memory cache is used instead.

Implement caching strategies in Django

You’ll now implement several caching techniques in Django: caching expensive database queries, template fragments, entire views, and sessions.

Cache expensive database queries

Memcache is often used to cache expensive database queries. Our simple task list app doesn’t include any resource-intensive queries, but for the sake of learning, let’s imagine that getting all tasks from the database is an expensive operation.

The code to fetch the task list from the database can be modified to check the cache first. In myapp/views.py, update the following:

# myapp/views.py
# ...
from django.core.cache import cache
import time

TASKS_KEY = 'tasks.all'

def index(request):
    tasks = cache.get(TASKS_KEY)
    if not tasks:
        time.sleep(2) # simulate a slow query
        tasks = Task.objects.order_by('id')
        cache.set(TASKS_KEY, tasks)
    context = {'tasks': tasks}
    return render(request, 'index.html', context)

# ...

The above code first checks the cache to see if the tasks.all key exists in the cache. A database query is executed, and the result is cached if it does not. Subsequent page loads will use the cached value instead of querying the database. The time.sleep(2) exists to simulate a slow query.

Redeploy the app to Render with:

(venv) $ git add .
(venv) $ git commit -m 'Add query caching'
(venv) $ git push

View Memcache statistics

To help demystify Memcache caching operations, it’s helpful to visualize what’s going on under the hood.

Though very cumbersome, one way to do that is to telnet into a Memcached server and run the stats command to see changes as operations are performed on your cache.

With MemCachier, however, you get an analytics dashboard that displays your cache’s statistics so you can monitor performance and troubleshoot issues quickly and easily.

To open your MemCachier analytics dashboard, log in to your MemCachier account, click Caches, then click the Analytics button for your cache.

Add a task, and the following should happen:

1. cache.get(TASKS_KEY) is called, but the task list is not yet in the cache (get misses +1)
2. cache.set(TASKS_KEY, tasks) then stores the task list in the cache (Set Cmds +1)

Now, if you refresh the page:

1. cache.get(TASKS_KEY) successfully gets the task list from the cache (get hits +1)

The cache is working, but there is still a significant problem. Add a new task and see what happens. No new task appears on the current tasks list! The new task was created in the database, but the app serves the stale task list from the cache.

Clear stale data

There are many techniques for dealing with an out-of-date cache. You’ll learn about four options:

• Time-based expiration.
• Deleting the cached value.
• Key-based expiration.
• Updating the cached value.

1. Time-based expiration

The easiest way to make sure the cache does not get stale is by setting an expiration time. The cache.set method can take an optional third argument, which is the time in seconds that an item should stay in the cache. If this option is not specified, it defaults to the TIMEOUT argument of the appropriate backend in the CACHES setting in settings.py.

You could modify the cache.set method to look like this, caching the item for 5 seconds:

cache.set(TASKS_KEY, tasks, 5)

This functionality only works when it is known how long the cached value is valid. In our case, however, the cache gets stale upon adding or removing a task, so this technique is inappropriate.

2. Delete the cached value

A straightforward strategy is to invalidate the tasks.all key when you know the cache is out of date—namely, to modify the add and remove views to delete the tasks.all key:

# myapp/views.py
# ...
def add(request):
    if 'name' in request.POST:
        task = Task(name=request.POST['name'], notes=request.POST['notes'])
        task.save()
        cache.delete(TASKS_KEY)
        return redirect('/')
    return render(request, 'add.html')

def remove(request):
    task = Task.objects.get(id=request.POST['id'])
    if task:
        task.delete()
        cache.delete(TASKS_KEY)
    return redirect('/')

Now, whenever a task is added or removed, the cached tasks will be deleted and re-cached the next time the task list is viewed.

3. Key-based expiration

Another technique to invalidate stale data is to change the key:

# myapp/views.py
# ...
import random
import string

def _hash(size=16, chars=string.ascii_letters + string.digits):
    """
    Generate a random string of specified length and character set.

    Args:
        size (int): Length of the string to be generated. Default value is 16.
        chars (str): Set of characters to be used for generating the string. Default is the set of ASCII letters and digits.

    Returns:
        str: A random string of specified length and character set.
    """
    # Use random.choice() to randomly select characters from the specified character set
    # Repeat this process for the specified length of the string
    # Use join() to concatenate all the randomly selected characters into a single string
    return ''.join(random.choice(chars) for _ in range(size))

def _new_tasks_key():
    return 'tasks.all.' + _hash()

TASKS_KEY = _new_tasks_key()

# ...

def add(request):
    if 'name' in request.POST:
        task = Task(name=request.POST['name'], notes=request.POST['notes'])
        task.save()
        global TASKS_KEY
        TASKS_KEY = _new_tasks_key()
        return redirect('/')
    return render(request, 'add.html')

def remove(request):
    task = Task.objects.get(id=request.POST['id'])
    if task:
        task.delete()
        global TASKS_KEY
        TASKS_KEY = _new_tasks_key()
    return redirect('/')

The upside of key-based expiration is that you do not have to interact with the cache to expire the value. The LRU (Least Recently Used) eviction of Memcache will eventually clean out the old keys. When the memory limit is reached, LRU eviction is used in Memcache to remove the least recently used items from cache memory.

4. Update the cached value

Instead of invalidating the key, a cached item’s value can be updated:

# myapp/views.py
# ...

def add(request):
    if 'name' in request.POST:
        task = Task(name=request.POST['name'], notes=request.POST['notes'])
        task.save()
        cache.set(TASKS_KEY, Task.objects.order_by('id'))
        return redirect('/')
    return render(request, 'add.html')

def remove(request):
    task = Task.objects.get(id=request.POST['id'])
    if task:
        task.delete()
        cache.set(TASKS_KEY, Task.objects.order_by('id'))
    return redirect('/')

This approach is more straightforward to implement than key-based expiration but as performant. And it means the task list is immediately re-cached, so the next time it is loaded, database query and cache.set calls are unnecessary.

Choose option 2, 3, or 4 to ensure the cache is never out-of-date. I’m using option 4, updating the cached value, for this tutorial.

As usual, commit your changes and redeploy the app:

(venv) $ git add .
(venv) $ git commit -m 'Add query caching invalidation'
(venv) $ git push

Now, when you add or remove a task, all the tasks you’ve added since implementing caching will appear.

If you’ve implemented option 4 to clear stale cache data, each time you add or remove a task, the following should happen:

1. cache.set(TASKS_KEY, ...) updates the cached task list (Set Cmds +1)
2. cache.get(TASKS_KEY) fetches the updated cached task list (get hits +1)

Cache template fragments

Django allows you to cache template fragments. This is similar to snippet caching in Flask. Add {% load cache %} to the top of your template to enable fragment caching.

To cache a rendered set of task items, we use a {% cache <timeout> <key> <optional arguments> %} statement in myapp/templates/index.html:

<!-- myapp/templates/index.html -->
{% load cache %}
<!DOCTYPE html>
<!-- ... -->
  <ul>
    {% for task in tasks %}
      {% cache None taskfragment task.id %}
      <li>
        <a href="/detail/{{ task.id }}">{{ task.name }}</a>
      </li>
      {% endcache %}
    {% endfor %}
  </ul>
<!-- ... -->
</html>

Here the timeout is None and the key is taskfragment. Because this fragment has dynamic data (the task ID and name), you pass an additional argument task.id to uniquely identify each fragment.

As long as task IDs are never reused, this is all there is to cache fragments in Django. If you use a database that does reuse IDs, you must delete the fragment when its respective task is deleted. You can do this by adding the following code to the remove view in myapp/views.py:

# myapp/views.py
# ...
from django.core.cache.utils import make_template_fragment_key

# ...

def remove(request):
    task = Task.objects.get(id=request.POST['id'])
    if task:
        task.delete()
        # ...
        key = make_template_fragment_key('taskfragment', [request.POST['id']])
        cache.delete(key)
    return redirect('/')

Deploy your changes:

(venv) $ git add .
(venv) $ git commit -m 'Add fragment caching'
(venv) $ git push

Let’s see the effect of caching the fragments in our application. The next time you refresh the tasks list view, the following will happen:

1. Django checks if each fragment is in the cache. No fragments are currently cached (get misses +1 per task)
2. Django stores each fragment in the cache (Set Cmds +1 per task)

And on subsequent page refreshes:

1. Django checks if each fragment is in the cache. All fragments are now cached (get hits +1 per task)

Cache views

We can go one step further and cache the output of entire views in addition to fragments. This should be done carefully because it can result in unintended side effects if a view frequently changes or contains forms with CSRF tokens, as explained earlier. The task list in our app changes each time a task is added or deleted, so the cached view must be cleared after both actions.

You can cache the task list view with the @cache_page(<timeout>) decorator in myapp/views.py:

# myapp/views.py
# ...
from django.views.decorators.cache import cache_page

# ...

@cache_page(None)
def index(request):
    # ...

# ...

We must delete the cached view whenever we add or remove a task. This is not trivial. We need to learn the key when the view is cached to delete it:

# myapp/views.py
# ...
from django.utils.cache import learn_cache_key

# ...

VIEW_KEY = ''

@cache_page(None)
def index(request):
    # ...
    context = {'tasks': tasks}
    response = render(request, 'index.html', context)
    global VIEW_KEY
    VIEW_KEY = learn_cache_key(request, response)
    return response

def add(request):
    if 'name' in request.POST:
        # ...
        cache.delete(VIEW_KEY)
        return redirect('/')
    return render(request, 'add.html')

def remove(request):
    # ...
    if task:
        # ...
        cache.delete(VIEW_KEY)
    return redirect('/')

We use the Django cache util learn_cache_key to get the key when the view is cached. Then, delete the item from the cache when adding or deleting a task.

Deploy the changes:

(venv) $ git add .
(venv) $ git commit -m 'Add view caching'
(venv) $ git push

@cache_page stores the page in two items, one for headers and one for the page. The keys for those items look something like this:

# Headers
:1:views.decorators.cache.cache_header..7822c3f267e735391510ed308a3a0176.en-us.UTC

# Page
:1:views.decorators.cache.cache_page..GET.7822c3f267e735391510ed308a3a0176.d41d8cd98f00b204e9800998ecf8427e.en-us.UTC

On the first refresh, the following will happen:

1. @cache_page tries to fetch the header item, but it’s not yet cached (get misses +1)
2. @cache_page stores the header item in the cache (Set Cmds +1)
3. @cache_page stores the page item in the cache (Set Cmds +1)
4. learn_cache_key() also stores the header item in the cache (Set Cmds +1)

On subsequent page loads, the following happens:

1. @cache_page fetches the header item from the cache (get hits +1)
2. @cache_page fetches the page item from the cache (get hits +1)

Note view caching does not make the caching of expensive operations or template fragments redundant. It’s good practice to cache smaller operations within larger cached operations or smaller fragments within larger fragments. This technique (called Russian doll caching) helps with performance if a larger operation, fragment, or view is removed from the cache because the building blocks do not have to be recreated from scratch.

Advanced Memcache Debugging

With three caching strategies in place (query, fragment, and view caching), it can be challenging to determine precisely what cache activity is happening.

Our analytics dashboard also comes with advanced cache Introspection features (Prefixes, Keys, and Recent Requests), available starting with our Advanced plan (1 GB+). These tools can prove invaluable for debugging, troubleshooting, and ensuring things are working as expected.

When writing this tutorial, I regularly referred to the Recent Requests view to verify the executed cache commands. That view shows the last 100 lines of logs for each of your cache servers.

Here are some logged commands you would expect to see with Django, with my comments added:

# Try to get tasks. "Key not found", not in cache (`get misses` +1)
2023-02-23T14:18:12Z GETK Key not found 12B  :1:tasks.all
# Store tasks. "OK", stored successfully (`Set Cmds` +1)
2023-02-23T14:18:14Z SET  OK            1KB  :1:tasks.all

# Try get fragment. Not in cache (`get misses` +1)
2023-02-23T14:39:17Z GETK Key not found 63B  :1:template.cache.taskfragment.691513953ea19e01a0e7881a339ce106
# Store fragment. Stored successfully (`Set Cmds` +1)
2023-02-23T14:39:17Z SET  OK            144B :1:template.cache.taskfragment.691513953ea19e01a0e7881a339ce106

# Try get view. Not in cache (`get misses` +1)
2023-02-23T14:18:12Z GETK Key not found 82B  :1:views.decorators.cache.cache_header..7822c3f267e735391510ed308a3a0176.en-us.UTC
# Store view header. Stored successfully (`Set Cmds` +1)
2023-02-23T14:18:14Z SET  OK            95B  :1:views.decorators.cache.cache_header..7822c3f267e735391510ed308a3a0176.en-us.UTC
# Store view page. Stored successfully (`Set Cmds` +1)
2023-02-23T14:18:14Z SET  OK            1KB  :1:views.decorators.cache.cache_page..GET.7822c3f267e735391510ed308a3a0176.d41d8cd98f00b204e9800998ecf8427e.en-us.UTC

Use Memcache for session storage

Memcache works well for storing information for short-lived sessions that time out. However, because Memcache is a cache and therefore not persistent, long-lived sessions are better suited to permanent storage options, such as your database.

For short-lived sessions configure SESSION_ENGINE to use the cache backend. Add the following to the end of myproject/settings.py:

# myproject/settings.py
# ...
SESSION_ENGINE = 'django.contrib.sessions.backends.cache'

Django allows you to use a write-through cache backed by a database for long-lived sessions. This is the best option for performance while guaranteeing persistence. To use the write-through cache, configure the SESSION_ENGINE in myproject/settings.py like so:

SESSION_ENGINE = 'django.contrib.sessions.backends.cached_db'

For more information on how to use sessions in Django, please see the Django Session Documentation.

Clean up

Once you finish this tutorial and no longer need your app, you can delete your Web Service and Database from the Render dashboard. You could also delete the corresponding Blueprint from Blueprints.

You can also delete your MemCachier cache from the Caches dashboard if you no longer need it.

Further reading and resources

• MemCachier Django Documentation
• Render Django Documentation
• Django Caching Documentation