Matrix Builds with Automatic sbt-release

bpholt's picture

Following up on yesterday’s post, I ended up modifying the build process for the scala-aws-utils project. The two problems described in the “Caveats / Room for Improvement” section bothered me.

Protected branches help prevent accidental pushes to master, a backstop I didn’t want to lose. The switch from Java 7 to Java 8 for the Scala 2.10 and 2.11 artifacts wouldn’t have impacted our use of the library, but why unnecessarily restrict who can use those artifacts?

Enabling Matrix Builds

Prior to releasing with sbt-release, scala-aws-utils had a build matrix that looked like this:

With the excludes in place, this meant three builds ran:

Scala 2.10 Scala 2.11 Scala 2.12
Java 7
Java 8

The deploy provider would run in each build for any commits on the master branch.

To enable artifacts to be published from the Travis CI matrix builds, the version commits and release tag had to be moved out of the matrix builds and into a job that would run only once for each merge into master.

To accomplish this, I removed the deploy: provider I set up yesterday to run sbt-release, and replaced it with two new ones.

In the first section, the on block restricts the deploy to running on builds that are on the master branch, running for Scala 2.12, and match the specified custom condition, which checks that the committer name is not “Dwolla Bot,” the name we’ve assigned to the commits coming from sbt-release.

The custom condition is necessary to prevent builds that run on the sbt-release version commits from releasing again, causing an infinite loop. Previously this was prevented by adding [ci skip] to the commit message, but that technique would prevent the second deploy provider from running when we need it to.

The provider runs the release script, which prepares the local clone for release and then runs sbt release. The conditions ensure that the deploy only runs once per actual merge into master.

The second provider runs on tagged commits (which is the commit that’s technically being released, and has the correct version details) if the committer is named “Dwolla Bot.” This is a lightweight way to prevent someone from accidentally pushing a non-release tag to the repo and having it build.

In this provider, we don’t limit the deploy to Scala 2.12, and instead of running sbt release, it runs sbt publish. This results in three artifacts being published, corresponding to the build matrix above.

sbt release process

You may remember that previously sbt release was handling the publishing step, so running sbt publish separately would have been redundant. In this new configuration, the release process is modified to skip the publish step.

The commit message was also reverted to the default message, and cross-Scala releases were disabled.

Pushing to a Protected Branch

The process I described yesterday works fine for projects that don’t need matrix releases, but it still has the limitation of only working for GitHub repos that don’t restrict pushing to master. Without that restriction, though, the benefits of this whole project are limited by the fastidiousness of the team members with commit access. Since it’s always possible to commit human error, I wanted to see if we could get branch restrictions working again.

First, we need a collaborator whom we can allow to push to master. Go to the repo Settings tab and select Collaborators & teams.

Collaborators & teams

I have added our CI user dwolla-bot as a collaborator and given it Write access.

Next, to enable branch protections, go to the repo Settings tab and select Branches.

Branch Settings

Select master in the Choose a branch… select box, and configure the following options:

Branch restrictions

  • Protect this branch enables branch protections and makes available the other options
  • Restrict who can push to this branch allows us to add a user who can push to master. Only collaborators or team members can be added here, but since I added dwolla-bot as a collaborator earlier, I was able to add it here.

There are still a couple compromises here.

  • Require status checks to pass before merging and its child, Require branches to be up to date before merging are good best practices when using CI. But their enforcement relies on pull request reviewers ensuring the checks pass. Normally I would check the continuous-integration/travis-ci option here, but doing so prevents the version update commits created by sbt-release from being pushed.
  • Require pull request reviews before merging ensures someone other than the requester has looked at the proposed code prior to its being merged to master, if the requester otherwise would be able to merge the pull request. This is desirable, but again, enabling it prevents the sbt-release commits from being pushed.

It may be possible to make dwolla-bot an administrator of the repo and then disable Include administrators, but then human administrators can override the rules too. More tweaking and feedback is needed in this area.

Pushing Commits from dwolla-bot

In order to make GitHub aware that the sbt-release commits are being pushed by dwolla-bot (and it should therefore ignore the branch restrictions in place), we need to adjust the way git interacts with GitHub. An API token seems to be the way to go.

We already had a machine user set up, but if you don’t, sign up for a new GitHub account. (Hint: use incognito mode to sign up so your main account isn’t signed out, and if you use GMail, this may be a good time to take advantage of its support for plus addressing.)

In the CI account’s personal settings, go to the Personal access tokens tab, and click Generate new token. (You may have to enter the password again to enter GitHub’s sudo mode.)

Personal access tokens

Name the new token and grant it access to the repo scope. For our purposes, no other scopes are necessary, so click Generate token.

New token

GitHub will show you the new token. Make sure to write this down somewhere safe, because this is the only time you’ll be able to see it! Don’t worry—I’ve already revoked the token in the image above, but you should protect your new token, perhaps in a password manager.

Now that the token exists, let’s add it to the Travis CI secrets.

Once we add this to the .travis.yml file, this will make the API token available to the build in the GH_TOKEN environment variable. (Both the variable name and value are encrypted in the secure string.)

The value belongs in the env section, which used to look like this:

We can’t just add the new value directly, because it needs to be a global setting—we don’t want it to be another matrix dimension. Instead, we move the existing keys under a matrix section and add a global section for the new key.

With this in place, we can change the upstream remote from using ssh to use HTTPS with the API token for authentication:

git remote add release https://$GH_TOKEN@github.com/Dwolla/scala-aws-utils.git

Conclusion

With this setup, things work pretty well. We’re able to better protect the master branch against accidental modification, and we’re publishing artifacts with different JDK versions for different Scala versions.

There are still some tradeoffs.

  • We’re running extra builds for the snapshot commits (where sbt-release updates the version from the released version (e.g. 1.3.2) to the next snapshot version (e.g. 1.4.0-SNAPSHOT). These commits don’t get released and don’t contain any meaningful changes to the project, so their builds aren’t very valuable.

    They might be able to be skipped by using a custom commit message containing [ci skip] if the new version contains -SNAPSHOT. Something like this, maybe?

  • The master branch protection still isn’t perfect, and really only protects against accidental modification. I’m not sure how to address this right now, though—maybe one of my colleagues will have some ideas.