Build a Rails one-click-app on DigitalOcean and scale it with Memcache

Sascha ~ December 4, 2018

Tags: DigitalOcean, Rails, Tutorial

Update: One-click-apps are now Marketplace images.

This tutorial will walk you through the steps of creating a simple Rails One-Click application on DigitalOcean and then add Memcache to prevent or alleviate a performance bottleneck.

Adding caching to your web applications can drastically improve performance. The results of complex database queries, expensive calculations, or slow calls to external resources can be stored in Memcache that can be accessed via fast O(1) lookups. Even for small sites, Memcache can make page loads snappy and help future-proof your app.

In this guide we will create a contact list application and scale it with Memcache. The sample app in this guide can be found here.

Prerequisites

Before you complete the steps in this guide, make sure you have all of the following:

Familiarity with Ruby (and ideally some Rails).
A DigitalOcean account.
If you like managing DigitalOcean resource via the CLI, you need the doctl installed and configured.

Create a Rails One-Click application

To build an app we first need a droplet. Either go to your DigitalOcean dashboard and create one or launch one via the CLI:

$ doctl compute droplet create rails-memcache --image rails-18-04 --region nyc1 --size s-1vcpu-1gb --ssh-keys <KEY_FINGERPRINT>

Give the droplet a minute to come up and then look up its IP via the dashboard or by typing

$ doctl compute droplet list

Now you can login to your droplet via

$ ssh root@<DROPLET_IP>

and visit the page at http://<DROPLET_IP>/.

Switch to production

Before we start adding functionality, let’s create a production environment for our application.

[root]# sudo -i -u rails

Note: Ideally you open two terminals to log into your droplet. One stays in root for system related configurations and actions, and one uses the rails user to develop the app.

The application is located in the example directory:

[rails]$ cd example

If you want, you can change the name of the folder but then you will need to also change the paths referenced within /etc/systemd/system/rails.service and /etc/nginx/sites-enabled/rails.

To create a production environment let’s create a .env file to host our environment variables:

RAILS_ENV=production

Note: your droplet comes with nano or vim preinstalled. If you have never used an editor in a terminal before, I recommend you use nano for now. To create and edit the .env file type nano .env.

In order for our application to pick up the environment variables in our .env file, we need to reference it from the systemd unit file that starts the rails app. As root edit /etc/systemd/system/rails.service as follows:

[Unit]
# ...

[Service]
# ...
EnvironmentFile=/home/rails/example/.env

[Install]
# ...

To take effect, have systemd reload the unit files (as root):

[root]# systemctl daemon-reload

From now on, the rails app will run in the production environment. To make sure this works properly we need to precompile the assets (they will be served via nginx). To accomplish this, run the following in the example directory:

[rails]$ RAILS_ENV=production rails assets:precompile

Add contact list functionality

Use the Rails scaffold generator to create an interface for storing and viewing a simple directory of names and email addresses:

[rails]$ rails g scaffold contact name:string email:string
[rails]$ RAILS_ENV=production rails db:migrate

Edit config/routes.rb to set contacts#index as the root route,

# config/routes.rb
Rails.application.routes.draw do
  resources :contacts
  root :to => 'contacts#index'
end

Restart your app (as root):

[root]# systemctl restart rails.service

Visit the page in your browser via the droplets IP. Follow the “New Contact” link and create a few records.

Add caching to Rails

Memcache is an in-memory, distributed cache. Its primary API consists of two operations: SET(key, value) and GET(key). Memcache is like a hashmap (or dictionary) that is spread across multiple servers, where operations are still performed in constant time.

The most common use for Memcache is to cache the results of expensive database queries and HTML renders so that these expensive operations don’t need to happen over and over again.

Provision a Memcache

To use Memcache in Rails, you first need to provision an actual Memcached cache. You can easily get one for free from MemCachier. This allows you to just use a cache without having to setup and maintain actual Memcached servers yourself. Make sure to create the cache in the same region as your droplet is in.

There are three config variables you’ll need for your application to be able to connect to your cache: MEMCACHIER_SERVERS, MEMCACHIER_USERNAME, and MEMCACHIER_PASSWORD. Get them from your MemCachier dashboard and put them into the .env file:

#...
MEMCACHIER_USERNAME=<YOUR_USERNAME>
MEMCACHIER_PASSWORD=<YOUR_PASSWORD>
MEMCACHIER_SERVERS=<YOUR_SERVERS>

Now you are ready to use Memcache in your application.

Configure Rails with MemCachier

Rails requires the dalli gem in your Gemfile in order to connect the Memcache server:

gem 'dalli'

Install the new dependency:

[rails]$ bundle install

Now, configure the default Rails caching backend to use the cache store provided by dalli and connect to MemCachier by modifying config/environments/production.rb to include:

# config/environments/production.rb
config.cache_store = :mem_cache_store,
                    (ENV["MEMCACHIER_SERVERS"] || "").split(","),
                    {:username => ENV["MEMCACHIER_USERNAME"],
                     :password => ENV["MEMCACHIER_PASSWORD"],
                     :failover => true,
                     :socket_timeout => 1.5,
                     :socket_failure_delay => 0.2,
                     :down_retry_delay => 60
                    }

To make it easier to see how this example works, temporarily turn off built-in caching (we will turn it on again later in this tutorial):

# config/environments/production.rb
config.action_controller.perform_caching = false

Cache expensive database queries

The code in your contacts controller (app/controllers/contacts_controller.rb) looks something like this:

# app/controllers/contacts_controller.rb
def index
    @contacts = Contact.all
end

Every time /contacts is requested, the index method is executed, and a database query to fetch all of the records in the contacts table is run.

When the table is small and request volume is low, this isn’t much of an issue, but as your database and user volume grow, queries like these can impact the performance of your app. Let’s cache the results of Contact.all so that a database query isn’t run every time this page is visited.

The Rails.cache.fetch method takes a key argument and a block. If the key is present, then the corresponding value is returned. If not, the block is executed and the value is stored with the given key and then returned.

In app/models/contact.rb, add the following method to the Contact class:

# app/models/contact.rb
def self.all_cached
  Rails.cache.fetch('Contact.all') { all.to_a }
end

In app/controllers/contacts_controller.rb change

# app/controllers/contacts_controller.rb
@contacts = Contact.all

# app/controllers/contacts_controller.rb
@contacts = Contact.all_cached

Note that we cache all.to_a instead of all. This is because since Rails 4 Model.all is executed lazily and you need to convert Contact.all into an array with to_a in order to cache the actual contacts.

To see what is going on under the hood, let’s also display some statistics on the index page. Add the following line to the index method in app/controllers/contacts_controller.rb:

# app/controllers/contacts_controller.rb
@stats = Rails.cache.stats.first.last

And add the following markup to the bottom of app/views/contacts/index.html.erb:

<!-- app/views/contacts/index.html.erb -->
<h1>Cache Stats</h1>

<table>
  <tr>
    <th>Metric</th>
    <th>Value</th>
  </tr>
  <tr>
    <td>Cache hits:</td>
    <td><%= @stats['get_hits'] %></td>
  </tr>
  <tr>
    <td>Cache misses:</td>
    <td><%= @stats['get_misses'] %></td>
  </tr>
  <tr>
    <td>Cache flushes:</td>
    <td><%= @stats['cmd_flush'] %></td>
  </tr>
</table>

Restart the page as root:

[root]# systemctl restart rails.service

Refresh the /contacts page and you’ll see “Cache misses: 1”. This is because you attempted to fetch the 'Contact.all' key, but it wasn’t present. Refresh again and you’ll now see “Cache hits: 1”. This time the 'Contact.all' key was present because it was stored during your previous request.

Expiring the cache

Now that Contact.all is cached, what happens when that table changes? Try adding a new contact and returning to the listing page. You’ll see that your new contact isn’t displayed. Since Contact.all is cached, the old value is still being served. You need a way of expiring cache values when something changes. This can be accomplished with filters in the Contact model.

Add the following code to app/models/contact.rb to the Contact class:

# app/models/contact.rb
class Contact < ApplicationRecord
  after_save    :expire_contact_all_cache
  after_destroy :expire_contact_all_cache

  def expire_contact_all_cache
    Rails.cache.delete('Contact.all')
  end

  #...

end

Restart the app as root and try again:

[root]# systemctl restart rails.service

Now you can see that every time you save (create or update) or destroy a contact, the Contact.all cache key is deleted. Every time you make one of these changes and return to /contacts, you should see the “Cache misses” count get incremented by 1.

Built-in Rails caching

The examples above explain how to fetch and expire caches explicitly. Conveniently, Rails builds in much of this functionality for you. By setting

# config/environments/production.rb
config.action_controller.perform_caching = true

in config/environments/production.rb Rails allows you to do fragment, action, and page caching.

Here we just briefly introduce these caching techniques. For more details and other techniques such as russian doll caching, please refer to the Rails Guide on Caching.

Fragment caching

Pages in Rails are generally built from various components. These components can be cached with fragment caching so they do not need to be rebuilt each time the page is requested.

Our /contacts page for example is built from contact components, each showing the name, the email, and 3 actions (show, edit, and destroy). We can cache these fragments by adding the following to @contacts.each loop in app/views/contacts/index.html.erb:

# app/views/contacts/index.html.erb
# ...
<% @contacts.each do |contact| %>
  <% cache contact do %>
    # ...
  <% end %>
<% end %>
# ...

Action caching

In addition to fragments, Rails can also cache the whole page with page and action caching. Page caching is more efficient as it allows a complete bypass of the Rails stack but it does not work for pages with before filters, such as authentication. Action caching stores objects and views similar to page caching, but it is served by the Rails stack.

To use action caching you need to add the actionpack-action_caching gem to your Gemfile and run bundle install:

gem 'actionpack-action_caching'

To cache the results of the show action, for example, add the following line in app/controllers/contacts_controller.rb:

# app/controllers/contacts_controller.rb
class ContactsController < ApplicationController
  caches_action :show
  # ...
end

For proper expiration, add the following line in both the update and destroy methods in contacts_controller.rb

# app/controllers/contacts_controller.rb
def update
  expire_action :action => :show
  # ...
end

def destroy
  expire_action :action => :show
  # ...
end

Note that even if you use action caching, fragment caching remains important. If a page expires, fragment caching makes sure the whole page does not have to be rebuilt from scratch but can use already cached fragments. This technique is similar to russian doll caching.

Memcache for session storage

Memcache works well for storing information for short-lived sessions that time out. However, because Memcache is a cache and therefore not persistent, long-lived sessions are better suited to permanent storage options, such as your database.

To use your cache for session storage create (Rails 5) or edit (Rails 3 and 4) the file config/initializers/session_store.rb to contain:

# config/initializers/session_store.rb
# Be sure to restart your server when you modify this file.
Rails.application.config.session_store :cache_store, key: '_memcache-example_session'