Adding New Quality Checks to the Build

update (2014-03-08): added reference to podcast time segments and clarified ambiguity

I just got around to listening to The Ship Show #37, and one of the topics they've discussed (47:00-48:00, 49:30-50:00) was adding new quality checks to the build. Or to be more precise, they've discussed talked about a case which exhibits the anti-pattern way of adding new checks to the build, namely:

  1. Find a new quality check (e.g.: let's not have compiler warnings)
  2. Get team buy in
  3. Modify the build to fail on these violations
  4. Have a broken build for the coming months - and have developers ignore real build failures (ah, it's broken only due to the new checks!)
  5. Abandon this new quality check so that a failed build actually means the build is actually broken

One way to avoid the above scenario is to focus on gradual improvement, i.e.: after steps #1 and #2 (which shouldn't be skipped!), in step #3, instead of a zero-tolerance policy, you set the limit for the new check violations at the current level - i.e.: the team commits to not making the situation any worse, and trying to improve it. Exceeding this limit will make the build fail, and thus there are no more false alarms - all build failures have to be taken seriously.

If your monitoring/build tool doesn't support "continuously decreasing limits" out of the box, you'll have to adjust this limit manually, let's say every morning. Be careful to only ever lower the limit though!

And there are other things you should be monitoring about your build too!

What do you think? I would love if you would leave a comment - drop me an email at, tell me on Twitter!

Posted on in monitoring, continuous integration, software by

Share this post if you liked it - on Digg, Facebook, Google+, reddit, or Twitter

Your email address