Automated Releases using sbt-release and Travis CI

bpholt's picture

At Dwolla, we have several Scala-based open source projects, each of which needs to be automatically tested and built. We have these projects set up to build using Travis CI, which the build results reporting back to GitHub for branches and pull requests. Merges to master are built and artifacts pushed to Bintray automatically as well.

We hadn’t automated the release process, though. Pull requests were responsible for updating the version information correctly, which could be tedious and somewhat error-prone. Concurrent pull requests needed to resolve versioning conflicts. Local development usually proceeds using a ‑SNAPSHOT version, so Ivy knows to overwrite any artifacts being created, but that means I have to remember to change the version in the project’s build definition before committing my changes.

Several Scala-based open source projects use the sbt-release to manage their release process. The plugin ensures no snapshot dependencies are being used, moves the project from a ‑SNAPSHOT version to a release version, commits and tags the version change, and builds and publishes artifacts. In the end, two commits are created (the release version and the next snapshot version).

Automating the Release Process

I couldn’t find any examples of automating the release process. Typically, it appears that projects require changes to go through a pull request, so they can be reviewed prior to reaching the master branch. Releases would be done manually on a commiter’s machine, with a human running sbt release. In practice, this means that team member must have authority to push directly to master, along with credentials to publish artifacts.

Internally at Dwolla, we do not allow pushing commits to master, nor do we allow publishing artifacts from team member’s machines. Both could result in unreviewed code reaching our production systems. A manual release step for our open source projects would be out of step with this principle.

Working from the sbt-release documentation and Steve Klabnik’s guide to automatically updating GitHub Pages, I pieced together the following to allow Travis CI to perform the release steps and push the commits back to GitHub. The rest of this post assumes a project that has been set up to publish to Bintray and is using the sbt-release plugin locally.

.travis.yml

Let’s start by reviewing the .travis.yml file from our scala-aws-utils project, which tells Travis CI how to build the repo.

This sets up a Scala build, with cross-Scala support for 2.10.6, 2.11.8, and 2.12.1. It uses Java 8 to perform all builds, and runs the test task to verify commits and pull requests.

Before deploys, it decrypts some secret configuration from the repo. The instructions for decryption will be given to us when we encrypt the file (see below). Once the archive is decrypted, it extracts the contents to specific locations on the build server’s filesystem.

To perform a deploy, it runs the .travis/release.sh in the repo, but only for commits on the master branch and when targeting Scala 2.12.

sbt-release is responsible for creating the commit with the version being released, so it’s easiest to use its support for cross-Scala builds. Previously, this project used a Travis build matrix to compile Scala 2.10 and 2.11 artifacts with Java 7, and Scala 2.12 with Java 8. The build matrix results in three separate builds for each push to master, so it doesn’t work with the default sbt-release process.

Encrypted Secrets

Where do the encrypted secrets come from, and what do they do? In this case, we have two secrets to manage. First, we need credentials to upload artifacts to Bintray. Second, we need an SSH private key to push commits to GitHub.

Because we need to manage multiple encrypted files, due to a limitation of Travis CI tooling, we need to create an archive of the files and encrypt that. This means we need to keep the plaintext version of these secrets somewhere safe, so we can re-run this step if/when we need to add to or update the secrets. I use a password manager for this: LastPass at work, and 1Password at home.

Obtaining Bintray credentials is out of scope for this post, but let’s talk about the GitHub deploy key. As far as I can tell, there are two reasonable options for allowing Travis CI to interact with a GitHub repository: GitHub API tokens and SSH deploy keys. (You can also use username & password authentication, but that’s unreasonable.)

Steve Klabnik’s guide uses API tokens, but I think they have some downsides for my use case. API tokens are tied to GitHub users, so they can interact with any repo the user can interact with, which is not in line with the principle of least access. Also, if the user leaves the project, builds will have to be updated to use someone else’s API token. There’s nothing wrong with the choice to use API tokens, though, so weigh the tradeoffs and use whichever makes the most sense for you.

SSH deploy keys are tied to repos, not users, so they don’t have the same downsides. If there are many repos that need to be set up, they come with a little extra overhead, but as you’ll see, it’s minimal.

First, we need to generate a keypair, which we can do locally:

I ran the ssh-keygen command with the -f argument, which specifies the name of the private key file. Because we’re going to encrypt the private key using Travis CI’s tools, I left the passphrase empty.

This results in two files, release-key and release-key.pub. release-key is the private key, and needs to be kept safe. release-key.pub is the public key, and can be safely published in plaintext.

Assuming the Bintray credentials are in the same directory as the keys, we next need to create the archive file:

bholt@dwolm046 ~/my-project $ cd .travis
bholt@dwolm046 ~/my-project/.travis $ tar czf secrets.tgz credentials release-key release-key.pub

Now that we have our archive, encrypt it using the Travis CI command line tool.

This encrypts the archive using a symmetric key known only to the Travis CI build servers, and writes the encrypted file to .travis/secrets.tgz.enc. I don’t typically use the --add option because I don’t want to accidentally overwrite any existing before_deploy section in our .travis.yml.

You backed up the secrets in a safe place, right? Let’s remove them from the repo so we don’t accidentally commit them.

bholt@dwolm046 ~/my-project $ rm .travis/secrets.tgz .travis/credentials .travis/release-key

GitHub Deploy Keys

Now that we’ve generated an SSH keypair and securely stored the private key in the encrypted secrets archive, let’s tell GitHub that the keypair is allowed to interact with our repository.

On the Settings tab of the repo, there will be a “Deploy keys” option on the left, where you can list any existing deploy keys and add new ones.

GitHub Deploy keys screen

Click “Add deploy key” and paste in the contents of release-key.pub. Make sure “Allow write access” is checked:

GitHub Add deploy key

Click “Add Key” to finish the process. GitHub will save the public key and allow the corresponding private key to interact with the repository.

The Release Script

By now, we have securely stored the secrets we need, and set up GitHub to allow the private key to interact with the repo. All that’s left is to script the process of preparing to release, and releasing itself.

So, reviewing the script paragraph by paragraph:

#!/usr/bin/env bash
set -o errexit -o nounset

This sets the file up as a bash script, and sets some flags to ensure the script doesn’t continue if an error occurs, or if we mistyped a variable name.

if [ "$TRAVIS_BRANCH" != "master" ]; then
  echo "Only the master branch will be released. This branch is $TRAVIS_BRANCH."
  exit 0
fi

This was an idea I saw in Steve Klabnik’s guide. It guards against accidentally releasing code from a branch other than master. (This step should be unnecessary, as long as nothing goes wrong at Travis CI.)

git config user.name "Dwolla Bot"
git config user.email "dev+dwolla-bot@dwolla.com"

When Travis CI clones the repository to build it, there is typically no need to identify who is doing so. In our case, though, we are going to be making commits, so we need to tell git to whom the commits should be attribute—in this case, a bot.

git remote add release git@github.com:Dwolla/scala-aws-utils.git
git fetch release

Travis CI clones the repository using HTTPS, meaning any upstream it sets will use HTTPS authentication. This can be made to work, but we want to use our SSH deploy key, so the script sets up a new remote using SSH, and fetches the list of branches and other metadata from the remote.

git clean -dxf
git checkout master
git branch --set-upstream-to=release/master

Travis CI clones the repository in a way that leaves us with a detached head. Again, this is typically not a problem, but we want to create commits that will be part of the master branch, so we reset the repository, checkout master, and set its upstream repository to the release remote we added earlier.

MASTER=$(git rev-parse HEAD)
if [ "$TRAVIS_COMMIT" != "$MASTER" ]; then
  echo "Checking out master set HEAD to $MASTER, but Travis was building $TRAVIS_COMMIT, so refusing to continue."
  exit 0
fi

This is another sanity check, in case master was updated between the time this build started and now. This could happen if several pull requests are merged at once, for example.

sbt clean "release with-defaults"

Finally, run sbt to release with the default settings, allowing it to proceed without further interaction by a human.

Final Step

Commit the changes to .travis.yml, .travis/release.sh, .travis/secrets.tgz.enc, and possibly sbt build definition files like build.sbt and push the commit to GitHub. If Travis CI knows about the repository, it will see the changes and start the build. On a branch other than master, it will skip the deploy step, so merge the commit into master and you should end up with artifacts published to Bintray, a release tag in GitHub, and an incremented version in the source code!

Caveats / Room for Improvement

As I mentioned earlier, this method doesn’t currently support Travis CI’s build matrix. I think it could be made to support the matrix (enabling e.g. Scala 2.10 builds with Java 6 or 7 while Scala 2.12 builds with Java 8) by modifying the default sbt-release process to exclude the publishArtifacts step. In that case, sbt-release would be responsible for making the version commits and tagging the appropriate commit. Then we could add a separate Travis CI deploy phase that only ran on tagged commits to actually publish the artifacts.

This approach also doesn't seem to work on repos that have master as a protected branch. I think a workaround for this would be to add a bot user to GitHub, authorize it to push to the protected branch, and either use its API key or SSH key to push.

Update: See my followup post for how I dealt with these issues!