Operating Trunk Based Development at Scale

Sevatec Code branching out in tree shape

All developers at most any level of skill and experience are familiar with Source Control tools and how to check out code, perform commits, merge code through pull requests and participate in code reviews. This is software development 101. But knowing the concepts and the techniques doesn’t mean you understand the best practices for utilizing these techniques to deliver large-scale development projects to production continuously.

Over the years, software engineering has matured from the casual practice of maintaining code in source control (like just storing some data in a database) to managing and controlling the actual coding process throughout the development.  Source control is now a primary process for any development effort and central to how software is created. The standard practice of code management under source control is Branching. Branching for features, individual developers, versions, and several other aspects related to coding creates distinct separation from the Master Branch. Frank Compagner from Guerrilla Games stated’ “Branches create distance between developers, and we do not want that.”  The problem with distance is that late stage merges create unforeseen or unexpected errors, hide duplication of work until the merge occurs, and issues around incompatibility will not be visible until too late.

To resolve these issues, new branching techniques have been developed. GitLabs, who has employed Git services to build a DevOps platform, created Git Flow. Flow uses a Development Branch to centralize code control. However, this still creates a complicated workflow as seen in the diagram below.  Multiple feature branches are merged into the development branch and when scanned, tested and approved, pushed to a release branch. The release branch continues to receive updates until a mature state is reached that constitutes a version release. This release is merged into the Master Branch that is deployed to production.

Although this method creates greater control over the work performed, it also impedes the speed of development. Long living feature branches will end up imposing many of the problems associated with distance and can develop a culture of micromanagement by the software architect.

So enters Trunk-based Development (TBD). TBD eliminates the speed issue and allows developers to be more autonomous although it does require more senior development staff.  But, maximizing the value of DevSecOps strives to build teams with more Full Stack Developers, so TBD becomes the method of choice for source control management. TBD uses a central Master Branch as the source of truth for the development as show below.

The one primary advantage of TBD is that the development team is always Release Ready. This supports management’s need for adaptive and flexible response to changing business demands by being able to incorporate new features quickly and deliver to deployment. TBD also supports teams operating at different cadences by allowing direct deployment from the Master branch for fast, adaptive teams while a Release Branch can be used to manage more complex, slower deployment teams.

To operate at Scale, TBD is best performed with short lived, feature branches tied to a developer’s work that only live for a couple days before merging back with the master trunk.  When code and quality review processes and automated scanning techniques are use during pull requests and merges, issues and defects can be caught earlier removing the impacts to continuous delivery later in the development cycle, especially during automated testing regimes.

Supporting multiple commits to the master branch per day makes for a more effective Continuous Integration process and ensures the codebase can be released on demand. Key advantages of TBD are quicker iterations, minimized errors, rapid deployment, and efficient management of multiple team development projects working towards a core deployment. Sevatec’s implementation of TBD also removes some of the disadvantages through our Governed Resource Management [link] that applies the same level of control you get from Git Flow without the degradation in speed.

With the ability to support multiple cadences of different development teams at scale through feature branch management, the ability to always deploy to production when necessary, and the more autonomous nature of work allowing developers to be more creative promotes a culture of Agile and DevSecOps best practices that leads to high quality modern software solutions.