Importance of a Proper Deploy Process: What We’ve Learned

As a provider for third party javascript, we know that our customers rely on, and trust that our embedded script be essentially invisible. We should not affect your load times. Obviously, we should not affect the functionality of your site at all.

Unfortunately, last night we broke that trust for a subset of our users that use the hash in a URL for application routing. A developmental branch of our tracking script was mistakenly pushed out, and temporarily created issues for a handful of our clients. This is absolutely unacceptable and we accept full responsibility. New features in the tracking script are always opt-in, so that your tracking behavior will not change unless you tell it to. Unfortunately this branch was still in development.

To prevent this from ever happening again, we are setting up better safeguards. Here’s what we’re doing.

Our Current Deploy Process

Our build and deploy process is currently handled by Grunt, and we keep master deployable. It runs a linter, checks unit tests, minifies/concats, and then uploads the file to our CDN. The current weakness in this is that the deploy can be run from any branch and should be restricted to master.

Expanding Our Testing

We have unit tests, but because tests haven’t been written for the new features, the unit tests were passing in our development branch. In the future, we’ll be following a more TDD approach and have failing tests written in before working on a feature.

Ensuring More Visibility When Deploying

Currently, there are no notifications when we deploy. We’re adding detailed notifications to our HipChat when deploys are done so that the entire team is aware of what’s happening.

As much as we wish we could, we can’t undo our mistake. What we can do is learn, and fix our deploy process so that it never occurs again. We’re deeply sorry for the inconvenience that this has caused our affected customers. If you have any concerns at all, don’t hesitate to email us at

Leave a Reply