After my recent talk at Cocoaheads about Shopify’s mobile development and testing/deployment process, I wanted to summarize the tools and open source projects I mentioned…

Rails

Regarding the deployment and testing of our Rails applications, we have some open source components:

  • Shipit – our production deployment tool
  • EJSON – for securely storing secrets in source code
  • Docker – containerization solution

iOS

For mobile development, our CI solution is BuildKite, which is a hybrid hosted CI solution. BuildKite provides the hosted management service for our automated testing, and handles all the webhook integration with GitHub and such. We provide the build machines which run BuildKite’s agent software.

Interesting features of BuildKite are that we can manage the build/test/deploy process using pipeline configuration files that are part of our source code management system, and also that BuildKite can operate with only minimal access to our GitHub repos – even able to operate without direct read access to our source code.

Our BuildKite agents are Mac mini computers hosted at a colo facility. Each one currently runs 2 macOS VMs with Xcode, fastlane, BuildKite’s agent and our other required tools.

The toolchain for our iOS/Android CI system includes some open source tools:

  • Packer – for automating the creating of our Mac VM images
  • Fastlane – tools for automating builds, tests, code signing, submission to beta services, etc..
  • Shenzhen – (mostly phased out) for automating builds to HockeyApp and other hosting solutions
  • FBSnapshot – library for iOS “snapshot” testing, taking screenshot of views and screens for testing
  • OCMock – library for mocking Objective-C methods and objects
  • OHHTTPStubs – library for mocking network requests using “canned” responses

We also use a hosted solution for deploying iOS simulator and Android emulator builds of our apps, which allows support and non-developers the ability to run copies of these apps via a web browser. This is thanks to a service called appetize.io.

Local development

We’re able to run a fairly complete copy of the Shopify software stack on our local machine. We used to use Vagrant and VMWare to run a complete Linux system on our Macs to develop with, but this solution had some issues.

Our current solution involves running all the shared services like Nginx, MySQL, Redis, and so on inside of a very lightweight Linux VM system, and running the Rails applications themselves directly on our Macs. We use a home-grown solution for managing our local system called dev and our VM system is called railgun.

Railgun is built on top of an open source VM solution xhyve that uses the built-in Hypervisor.framework, a built-in component of macOS that many people don’t know is there.

Mac tools

My last slide was a gratuitous list of tools I use, some of which are not that well known…

  • Tower – my favourite graphical Git client
  • Kaleidoscope – the best diff tools I’ve found, works much better than Git’s diff and also has super image support.
  • Trailer – Mac menu-bar app that gives quick access to GitHub issues and PRs and desktop notifications on updates
  • Charles Proxy – an invaluable tool for examining HTTP/HTTPS traffic
  • Paw – for querying and testing HTTP API services
  • SimPholders – menu-bar utility that helps you find your simulator builds in the Finder