What is a Merge Queue and Do You Need One?

Merge queues have been around for a while, but in recent months they’ve become a hot topic in software development. Part of this is due to Github’s recent announcement on making its merge queue feature generally available. However, the need for merge queues is continuing to rise amongst development teams. 

If you’ve worked in a large monorepo with a long laundry list of contributors, you’ve likely experienced the pain points that come with not having a merge queue. Broken builds, merge conflicts, and outdated pull requests are terms you’re all too familiar with. Depending on your setup, a merge queue can fix these issues and get you shipping faster.

In this article, we’ll cover everything you need to know about a merge queue. If you're less interested in the basics and want to jump straight into implementation, skip to the end of this article. 

What is a Merge Queue?

A merge queue is a tool that automates PR merges into your repo's main branch. They are a best practice for trunk-based development in repos with 10-1000 active engineers. Merge queues make sure your main branch is never broken. Broken, meaning: a service is not functioning correctly, your app segfaults, there are compilation errors, a unit test is failing, or anything else is going wrong.

There are two common ways main branches "break":

1. A PR was branched off of on an old commit of main, and when merged into main it no longer functions with the latest state of the repo.

2. Two (or more) PRs recently merged and worked independently, but combining their changes breaks something.

Either way, the more pull request velocity you have in a repo, the more often issues arise. These issues are called logical merge conflicts. A merge queue handles this problem by testing PRs queued to be merged in combination, based on the latest main, and only if the extra combination testing passes do they merge.

Simply put, merge queues quietly manage the chaos of code changes that often come up in large teams or monorepos. They're all about organizing the flood of contributions into a neat, orderly process that doesn't trip over itself.

Why Do You Need a Merge Queue?

If you’re an individual developer in a simple codebase or a small team in a repo, you likely do not need a merge queue. However, merge queues are a great fit for companies who work in a monorepo or have multiple teams working in one. If your codebase often feels like a free-for-all, where code clashes and quality control headaches are part of the daily grind then you should consider implementing a merge queue. 

Ultimately, deciding whether you need one boils down to your unique situation. If you're on the fence, we recommend considering the following questions.

• Do you work in a monorepo or manage a large codebase with multiple teams? 

• Does your team use trunk-based-development? 

• Are merge conflicts and integration headaches slowing your team down consistently?

• Is collaboration among teams leading to bottlenecks?

• Do you have existing systems in place that consistently regress after multiple “fixes”?

• How much of your team’s time is spent on building vs. dealing with the drudgery of getting their code to production? 

If you answered yes to multiple questions above, we highly recommend looking into a merge queue. 

Managing a Merge Queue with Trunk Merge

Trunk Merge is a sophisticated merge queue that prevents broken builds on your main branch without sacrificing dev velocity. It's a GitHub bot and a web app that devs use to merge pull requests. Trunk Merge can use any CI provider as long as you use GitHub for your repo hosting. Merge also ensures that every PR is tested against the latest commit on your main branch so you’ll never have to worry about the freshness of your CI results.

A common downside with merge queues is teams often sacrifice velocity as PRs wait in the queue. This isn’t an issue with Trunk Merge as we have two queue modes you can choose from depending on how you’d like to prioritize velocity: single and parallel mode. Single mode is a great way to start, and parallel mode is a great way to scale a repo to 10s or 100s of active developers.

In Single mode, Trunk Merge acts like a typical queue: first in, first out. All PRs are tested and merged in the order they arrived. It will still test many combinations of enqueued PRs at once against each other regardless of whether two PRs are unrelated, it will test them against each other. This is a simple and effective way to start using Trunk Merge.

In Parallel mode, Trunk knows which PRs are related and which are unrelated and can function effectively as having many merge queues in the same repo, queueing only related PRs on top of one another. Trunk knows the relationship between PRs by you sending it to us. We call these Impacted Targets and it can be information pulled from build systems like Bazel, Nx, or Turborepo. It can also be defined by a set of glob file matching patterns.

Setting up a Merge Queue with Trunk Merge

Setting up your first merge queue with Trunk Merge is relatively straightforward. Feel free to start using the guidelines below for setup or navigate to our docs.

Connect your Trunk Organization to GitHub

First, sign up at app.trunk.io and create a Trunk organization. Then connect your organization to your GitHub Repositories. Select the repository you would like to use and click Get Started.

Setup Trunk Merge

Go to the Merge tab. You will see the Set Up Trunk Merge page. From there, you can specify:

1. The name of the branch that Trunk Merge should help manage merging PRs into.

2. The number of pull requests that Merge can test at the same time.

3. The mode that Trunk Merge will start in.

Define Required Status Checks For Testing

Trunk needs to know which status checks must pass while testing pull requests in the queue before it can merge a PR into your branch. Merge can pick up this list of required statuses in one of two ways:

1. From the list of Require status checks to pass before merging specified in the GitHub branch protection rule for your merge branch. This is particularly useful if you want Merge to track the same status checks that must pass before GitHub can merge a PR.

2. From the merge.required_statuses section of your .trunk/trunk.yaml. Choose this setup if you want the list of status checks that must pass before Merge can merge a PR to be different from the required status checks that must pass as a part of GitHub branch protection.

To specify using the .trunk/trunk.yaml file, set the merge.required_statuses to the name(s) of the GitHub status checks or jobs that must pass:

1version: 0.1 
2cli:
3  version: 1.16.0
4merge: 
5  required_statuses:
6    - Trunk Check 
7    - Unit tests & test coverage
8    # Add more required statuses here
9

To use GitHub branch protection instead follow GitHub's instructions for requiring status checks.

Configure a CI workflow to run on via Trunk Merge

Trunk Merge creates branches with the prefix trunk-merge/ in order to test PRs. To ensure the required statuses Merge should gate on get triggered when it tests PRs, your CI provider must be configured to run the status checks you care about whenever a branch with that prefix is pushed to.

For GitHub Actions, that'll mean setting up a push-triggered workflow, filtered to trunk-merge/** branches, like so:

1name: Run Required Checks
2run-name: PR Checks for ${{ github.ref_name }}
3
4# Trigger jobs whenever Trunk Merge tests a PR using a `trunk-merge/` branch
5on:
6  push:
7    branches:
8      - trunk-merge/**
9
10jobs:
11  trunk_check:
12    runs-on: ubuntu-latest
13    # "Trunk Check" is specified in merge.required_status above
14    name: Trunk Check
15    steps:
16      - name: Checkout
17        uses: actions/checkout@v3
18
19  unit_tests:
20    runs-on: ubuntu-latest
21    # "Unit tests & test coverage" is specified in merge.required_status above
22    name: Unit tests & test coverage
23    steps:
24      - name: Checkout
25        uses: actions/checkout@v3
26
27    # Add more steps here..    

Now you are ready to submit your first PR.

Submit Pull Requests

Try making a simple change on a branch and submit it as PR in GitHub.

Now trigger Trunk Merge to process this PR using either a comment on the PR in GitHub or using the Trunk CLI. If you have any problems with merge queueing PRs, take a look at the branch protection docs.

Pull Request Processing

Once a PR is submitted to the merge queue, it will start as Not Ready until all of the required conditions to submit it are met. These conditions are configurable, but it’s often required the CI jobs pass and a code reviewer approves the PR. Once ready, the merge queue will pick it up and run the tests. Once the tests pass, the PR may still need to wait for upstream PRs in the queue to finish their testing. Once the remaining upstream PRs are complete, the PR will be merged and then removed from the merge queue. If a PR fails or is canceled then it will go to the failed or canceled state. Read more about PR States.

Next Steps for your Merge Queue

Now Trunk Merge is setup with your repo. Whenever a PR is pushed to your merge branch it will be safely tested and automatically merged when all tests pass, regardless of the order they were pushed in.

For next steps, you can configure parallel mode for potential performance gains, read how to cancel pull requests, and setup a Slack Integration. If you are using Bazel you may want to further customize it for parallel mode.