We need to be very careful when we write and deploy software. If we ship a critical bug or screw up a release, our cache might go down and take down our customers’ websites with it. Everything we do is focussed around offering a powerful, stable service to our customers. And we’ve developed a software engineering process to encourage this.
We have several guidelines we follow when we’re writing and deploying software:
- Ship code when it’s finished and no sooner.
- Code review every diff unless you pair programmed.
- Communicate asynchronously when possible. Try not to interrupt others.
- Canary release when possible.
- Test and benchmark everything.
- Be an artist and write beautiful code.
- Support decisions with data and reason.
We also optimize for uninterrupted hacking: no meetings, no tracking tickets, no burn down charts, no sprints, no mandated work hours, no mandated work location, no vacation policy, and no deadlines.
All of this may seem counterintuitive. Usually careful code is written under heavy organization and process — think commercial airplane code. Yet our process has worked very well for us. Our recent outages weren’t caused by bad code or release mistakes. The code we write has thus far made its way safely to the memcache requests of our customers.
Obviously not breaking our service is very important to us. But so is our happiness. The culture we’ve adopted has given us incredible freedom to be happy. We like exploring programming languages and different designs and encodings. We care about the stack from the hardware level of NICs, caches and registers, up to assembly and higher level languages. We read assembly code one minute, Ruby the next, and then RFC’s on DNS and TCP/IP after that. We value our code and our knowledge, and we’ve chosen a culture that accommodates.
If you’re an engineer and you want to work in a hacker culture, you should come work with us at MemCachier — we’re hiring.
Image credit: hacking the Gibson