Reflections on Trunk-Based Development

In the past few years there has been an elevated level of buzz about trunk-based development. In its most extreme form it is perceived to be a branching model with just a single branch, the trunk, better known in Git as master or main. Feature branches are declared unnecessary and long-lived branches are seen as the absolute evil. Knowing what I know, it was easy for me to ignore such suggestions.

While working on SemVer Branching article, I came across Trunk-Based Development site by Paul Hammant, which presents more complete and certainly less extreme view of the approach. The site defines Trunk-Based Development as

A source-control branching model, where developers collaborate on code in a single branch called ‘trunk’, resist any pressure to create other long-lived development branches by employing documented techniques. They therefore avoid merge hell, do not break the build, and live happily ever after.

Now, “resist any pressure” does not mean “verboten” and even on Introduction page one can see on display optional feature and long-lived release branches. This is something I can live with: instead of absolutes we are now talking shades of gray. Moreover, the model I described in my article also uses optional feature and long-lived releng branches. That got me curious: what are the differences then?

So far, I was able to spot two: the lack of support for minor versions and the direction of the code propagation for bug fixes. Let’s look at each one of them.

I like the fact that Trunk-Based Development, as described, recognizes the need to maintain multiple versions of software. This certainly reflects the reality we all live in. The emphasis on the only one true supported “production” version of software is a special case, made popular by proliferation of web and mobile applications. The software universe as a whole is much bigger than this—hey, some software products are not even applications.

The desire to provide support for previous versions and to stabilize the code from the trunk a bit before releasing it, manifests itself in release branches. The description says that these branches “should not receive continued development work”. I have to deduce that receiving bug fixes in some shape or form may be OK, otherwise what is the purpose of all that?

Looking at the release branch naming pattern in the description, it may look like minor releases are supported: Rel 1.1.x branch yields versions 1.1, 1.1.1, 1.1.2 and so on. (Version number 1.1 can be normalized as 1.1.0 to make its format SemVer compatible.) I have to argue that minor releases in the SemVer sense of the word cannot possibly be supported by this branching model, unless all changes in the trunk support backward compatibility of the API.

Branches Rel 1.1.x and Rel 1.2.x originate from two distinct places in the trunk, and if we assume SemVer rules about version numbering, all changes on the trunk between these two commits must not break the API. I also have to treat “the API” a bit wider than just programmatic calls one piece of software makes to another. E.g. it should also maintain backward compatibility of the file formats it reads and writes, and it better not drop any user accessible functions either.

Can we always guarantee that with the techniques such as feature flags or branch by abstraction? Maybe, but I have my doubts. It is way easier for me to assume that sometimes backward compatibility is not there, even when the version number increment is minor. Google, the avid follower of Trunk-Based Development, seems to be honest about this fact. For example, the Google Chrome version on my machine as of this writing is 146.0.7680.178. Note the zero in the second position. It has been like this for the longest time.

Next, let’s look at the direction of the bug fix propagation. It is suggested that

The best practice for Trunk-Based Development teams is to reproduce the bug on the trunk, fix it there with a test, watch that be verified by the CI server, then cherry-pick that to the release branch and wait for a CI server focusing on the release branch to verify it there too. Yes, the CI pipeline that guards the trunk is going to be duplicated to guard active release branches too.

It so happens that the approach I use is exactly the opposite. I try to reproduce the bug with an automated test in the lowest supported version. Often, this will be the version in which the bug was reported but not necessarily. Then the production code is then changed to make the test pass. In the vast majority of cases, the test is a unit test and no CI server is in the picture. The CI build happens when I push to the releng branch that supports the fixed patch version.

Once I am satisfied with the fix, a sequence of regression avoidance merges is performed from the lowest version with the fix to the one being developed in the trunk (master) using the version order defined by SemVer. These will also trigger CI builds in the respective branches. Because the bug fix is the only change done to the releng branch, and because we do regression avoidance merges all the time, the bug fix commits will be the only changes present in this sequence.

A failure to reproduce the bug in the lowest supported version probably means that the version is not affected by the bug. This still requires me to go up the supported version list and see if the bug manifests itself in any of these versions. On the other hand, it is not all that unlikely to see that during the regression avoidance merges the code affected by the bug is no longer relevant or even present. Usually this manifests in a non-trivial production code conflict. The automated test is the ultimate verification though.

Why did I settle on this approach? Because figuring out what to cherry-pick hurts my brain, and I’ve missed commits in the past. This is not the end of the world and things get sorted out, but with merge I get the required change automatically. Also, branches tend to diverge from one release to the other. Arguably, it should be easier to reproduce the bug closer to the time when it was introduced to the code. For critical bugs time matters a lot, and every little thing helps.

One of the claims made by the site is that

You should do Trunk-Based Development instead of GitFlow and other branching models that feature multiple long-running branches.

No argument about GitFlow. The disagreement is obviously about “other branching models.” You see, while there are some differences in the way long-lived branches are handled in SemVer branching, but ultimately the version tags are assigned to releng/X.Y branches. This happens to be directly equivalent to the way the version tags are assigned in branch for release style in Trunk-Based Development. Moreover, the only changes committed to releng/X.Y are critical bug fixes. These branches do not receive “continued development work”.

So, there you have it. In my opinion this claim does not hold. I would not go as far as saying that if you use SemVer branching you also follow Trunk-Based Development.