Hard to understand systems

Admitting something is hard to understand is often very difficult for software engineers, as no matter how complex a system is, if you put enough effort in you will eventually gain insight.

The best systems (not just software) are often easy to understand. If systems are easy to understand, this means you can more easily reason about them; you can track down bugs more easily, add features and optimise more effectively. Easy to understand systems are often a collection of well defined components that we can treat as black boxes; you do not have to understand the details to understand the system as a whole.

There are notable exceptions; for example, the Linux kernel and its driver space is notoriously difficult to understand. That complexity has been introduced over time. Despite the fact that the kernel is monolithic, it started out with manageable levels of complexity and reached a critical mass. Now there are many enterprises and individuals invested in its success. That said, not attempting to simplify is very risky; how do we know how complicated is too complicated?

Whilst, “hard to understand” can be subjective, its rare that easy to understand systems just emerge; you need to put effort into making them easy to understand. There are many tools we can use to better reason about a system. Abstraction underlies most techniques for understanding. By hiding extraneous detail we make systems easier to understand.

Layers constrain the dependencies between components. Using layers makes an architecture easier to understand because it aims to limit connectivity between components. Each connection between components has a complexity cost. That said, not all architectures need to be layered. Some systems are simple enough that they don’t need layers. Some systems require a certain structure that is inherently complex, and cannot be fitted into layers; luckily they are the exception. Layers themselves have a complexity cost and should be used sparingly.

To help our understanding and communication, we need to develop a common language to talk about the system (a.k.a. ubiquitous language); well defined terms will help us. Naming is important. For example, what is an Entity? Its one of the building blocks of the system I work on, and last time I asked this question, I did not get very consistent replies from the team. It doesn’t matter if you’re using the term Entity to be something more specific than the real world, but it should still be consistent with the rest of the world. For instance, I’ve seen many examples classes named Cache in different codebases that don’t fit the real-world definition.

The principles behind the Agile manifesto say that the best architectures come from self-organising teams. This means that the best architectures are not dictated to the team; the team have to do the architecture themselves. In some self organising teams a senior member will naturally fill the position of architect and in other teams all team members will take collective responsibility for the architecture. I believe architecture is evolutionary and that you discover it on the way; in the process of building your system.