My Build System Manifesto

In my current role I’ve inherited responsibility for the build system for the products my company is working on. This roles has a pretty broad remit; I’m involved in lots of areas including how our modules our layed out, how code is generated, packaging, deployment and even how the build supports the software architecture

For context, the system I have inherited is quite complex; it’s a mixed Scala and C# (for Unity3d) project, with heaps of code generation tangled into the build. There is custom dependency management build on top of Maven/Ivy. Unity3d has its own build system. The build is used for both the components and the games themselves.

The previous approach was to create a monolithic build system with all of the components required to build games and libraries packaged up with the bottom layers of the software. Instead of a monolithic build, we should adopt a approach more like the Unix philosophy, that is, we should provide the tools that people can use to extend their own build instead of attempting the replace their entire process.

I’d like to rewrite this, but I don’t expect this to be easy; it’s a moving goal with many developers still extending this system. While thinking about this problem I wrote a list to describe what I think makes a good build system. By sharing this approach I wanted to get the other developers on board with the direction I’m going so that they don’t generate any more technical debt.

Hopefully this list is not controversial so I haven’t attempted to provide justification for each item. I’m hoping that the value of each should be obvious. These should serve as preferences or guidelines rather than rules. In some cases, there may be technical reasons that they’re not achievable.

  • I want to be able to check out the source that I need from a single repository. I want this to build and test with a single command. I want to go from nothing to a working development environment in very few steps.
  • I don’t want to have to open several IDEs; it should be optional. If I do have to open another, IDE the build should be self contained. I shouldn’t have to open the same IDE twice either. Using an IDE should be optional – command line should be an alternative.
  • I only want to check out the code that is relevant to me; if it’s not in the products I work on, I don’t need it. I want the area I work on to be “small”.
  • I want to continuously integrate with my team. I want to be able to branch and develop with the same abilities as master/develop.
  • Working across components should be rare, but if I need to work across components, I want to be able to do this with minimum fuss, all on my local machine. That is, with very few extra commands, without manual copying.
  • I want to be able to work on sub-projects like they are a top-level project. They should be self contained/independent. That is, I can build, test and publish with single commands. Their dependencies should be built. It’s just convenience that a sub project is grouped together with others.
  • I want to be able to leverage my skills for each platforms I develop on. For example, if I’m using C#, I want to use Visual Studio and MSBuild. I do not want to have to learn skills that are not transferable. I don’t want to learn a new platform if it isn’t essential to what I’m doing.
  • I want to be able to continuously deploy/integrate. I don’t want to have to deliver to other teams. I want to be able to add value myself. That is, teams should work on products, not components.

I wrote the list in a format a bit like user stories, where the users (or the “I”) is a developer.
Later, it struck me that this list reads like a manifesto. It is after all a declaration of my intentions.

I believe that the build is something critical to the success of a project. A build process done wrong can dramatically harm developer productivity.

This list should capture some of the most important points of an effective build system. I haven’t attempted to capture everything; they are from the context of where I am, there are some “obvious” points, such as reliable and reproducible that I haven’t documented here as we already have them.

This list should capture some guidelines/qualities that I often see overlooked. I might go into detail of each of the points in future posts.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s