Skip to content

The CI Tax Nobody Budgeted: Why We Moved to a Merge Queue Once AI Started Writing Our Code

· 11 min read · view as markdown

tags: dev-tools, ai-agents, architecture

At Selenios, for a long time the rule in our main repository was the one most teams have: a pull request can only merge if its branch is up to date with main and CI passed on that exact state. It is a good rule. It guarantees that what lands on main was tested as it will exist there.

It also has a cost that nobody puts in a budget, and this year that cost changed scale. Much of our code is now written together with coding agents. They write a lot of code, and to their credit, they write a lot of tests. Every time something merged, every other open pull request fell behind and had to be brought up to date, and every update re-ran the whole suite. When we measured it, about half of every pull request's CI minutes were going to that: not testing the change, but re-testing it because main had moved.

This is what we measured, why that cost grows faster than the code does, how we moved to GitHub's merge queue, and what changed in the first week. The short version: the queue removes the need to update a branch. It does not remove the habit.

More code, and far more tests

Between January and September of 2026, the merges into main per month grew 3.3 times. The test cases in the repository grew 74 times.

Growth since January 2026, as a multiple of January Log scale. Test cases in the repository grew 74 times between January and September 2026. Merges into main per month grew 3.3 times. ×1 ×3 ×10 ×30 ×100 Jan Feb Mar Apr May Jun Jul Aug Sep Jan: ×1.0 Feb: ×2.2 Mar: ×3.4 Apr: ×6.1 May: ×8.1 Jun: ×26.2 Jul: ×33.0 Aug: ×51.2 Sep: ×74.1 Test cases ×74 Jan: ×1.0 Feb: ×0.8 Mar: ×1.0 Apr: ×1.1 May: ×1.2 Jun: ×1.2 Jul: ×1.5 Aug: ×2.6 Sep: ×3.3 Merges / month ×3.3
Log scale, January = ×1. Test cases are it/test blocks in the repository at the end of each month; merges are squash commits on main. The jump in June is a deliberate push for test coverage across the whole codebase.

The second line is the one that matters for CI. Agents are good at writing tests. When you ask for a fix, they write a test that reproduces the bug. When you ask for a feature, they cover the edge cases a person would leave for later. That is exactly what we want, and it makes every CI run longer than the one before it. A suite that runs once per change is an investment. A suite that runs every time a neighbor merges is a tax.

The tax: keeping up with main

With the "require branches to be up to date" rule, a green pull request stops being green the moment anything else merges. Someone, or some agent, clicks "Update branch", the branch gets a merge commit from main, and the whole pipeline runs again: lint, types, build, the unit tests, the database-backed suites, and in our case a preview environment that is redeployed for that PR.

We looked at the pull requests opened after our last CI cleanup (drafts only run the fast checks) and merged before the queue:

  • 92 % were updated with main at least once, and on average almost three times.
  • Those updates were 51 % of the CI minutes of a pull request.
  • Each update also redeployed the PR's preview environment: about four in ten of all preview deploys were triggered by an update, not by a change to the code.
Where a pull request's CI minutes went before the merge queue For pull requests open under 4 hours, 4 to 24 hours, and over a day, about half of the CI minutes were runs triggered by updating the branch with main. Open < 4 h Runs for the PR's own commits: 50 Runs after updating with main: 50 100 50 % Open 4–24 h Runs for the PR's own commits: 75 Runs after updating with main: 94 169 55 % Open > 24 h Runs for the PR's own commits: 154 Runs after updating with main: 170 324 52 % Runs for the PR's own commits Runs after updating with main
CI minutes per pull request, indexed so that a PR open under 4 hours costs 100. Pull requests opened after our draft-CI change and merged before the queue. Whatever the lifetime, about half of the bill was re-testing main.

The share was about half no matter how long the pull request stayed open, which is the uncomfortable part. A pull request open for more than a day costs more than three times as much as one open under four hours, and half of that is still re-testing main.

Why AI makes this grow faster than the code

The number of updates a pull request needs is roughly the number of merges that land while it is open. That is where the arithmetic turns against you:

  • Merging three times as often means every open pull request falls behind three times as often.
  • There are also three times as many pull requests open to fall behind.
  • So the update runs grow roughly with the square of the merge rate, not linearly.
  • And each of those runs is longer, because the suite keeps growing.

There is one more effect, and it is specific to agents: they follow instructions. If a workflow says "bring the branch up to date before merging", an agent will do it every time, at three in the morning, for a branch that nobody is waiting on. In our repository, gh pr update-branch was on the list of commands agents could run without asking.

What a merge queue does instead

A merge queue turns the problem around. Instead of every pull request keeping itself up to date, the pull request says "I'm ready" and GitHub does the testing on top of main at merge time:

How a merge queue tests pull requests Each queue entry is built on main plus every entry ahead of it, CI runs once on that combination, and the tested commit is what gets merged. main Entry 1 main + A CI runs once Entry 2 main + A + B CI runs once Entry 3 main + A + B + C CI runs once Each entry is built on main plus the entries ahead of it. What merges is exactly the commit that was tested.
Up to three entries are tested in parallel. If one fails, it leaves the queue and the entries behind it are rebuilt without it.
  • Each entry in the queue is a temporary branch: main, plus the entries ahead of it, plus this pull request.
  • CI runs once on that combination. If it passes, that exact commit is what merges.
  • Several entries are tested in parallel, each one assuming the ones ahead will pass. If one fails, it leaves the queue and the ones behind are rebuilt without it.

The guarantee is the same as before, "what lands on main was tested as it exists there", but the cost is one run per pull request instead of one run per merge that happened while it was open.

The migration

The change in GitHub's settings is small. The work is making CI behave in a run that has no pull request attached. These were our changes.

1. Listen to the queue's event, and fix concurrency. Queue runs arrive as merge_group events. Our concurrency group was keyed on the PR number, which is empty in a queue run: every entry would have landed in the same group and cancelled the one ahead of it.

on:
  pull_request:
    branches: [ main ]
    types: [opened, synchronize, reopened, ready_for_review]
  merge_group:
    types: [checks_requested]

concurrency:
  # A merge_group run has no PR number: fall back to its own
  # gh-readonly-queue ref, which is unique per queue entry.
  group: ci-${{ github.event.pull_request.number || github.ref }}
  cancel-in-progress: true

2. Make "what changed" work in the queue. We skip frontend builds when a PR only touches the backend, using a path filter. The version we had pinned could not tell what a queue entry changed. Version 4 diffs the entry against its base (merge_group.base_sha..head_sha), so each entry only builds what it touches.

- uses: dorny/[email protected]
  id: filter
  with:
    filters: |
      backend:
        - 'backend/**'
        - 'pnpm-lock.yaml'
      frontend:
        - 'frontend/**'
        - 'pnpm-lock.yaml'

3. Decide what the queue runs. Draft pull requests in our CI only run the fast checks. A queue run has no draft flag, so the condition we use evaluates to "not a draft" and the queue always runs the full suite. That is what we want at merge time. Preview deploys, on the other hand, do not listen to the queue at all: a temporary queue branch needs no preview.

4. Turn it on. We moved the requirement into a repository ruleset: the CI summary check is required, "branches must be up to date" is off, and the queue is on.

{
  "type": "merge_queue",
  "parameters": {
    "merge_method": "SQUASH",
    "max_entries_to_build": 3,
    "max_entries_to_merge": 1,
    "grouping_strategy": "ALLGREEN",
    "check_response_timeout_minutes": 60
  }
}

We kept our classic branch protection, which carries the evidence our compliance controls rely on (code-owner review, no bypass), and only turned off its "up to date" requirement.

Before enabling it on main, we rehearsed on a throwaway branch with its own ruleset and a few dummy pull requests. Two things from that rehearsal were worth knowing:

  • When an entry fails, GitHub rebuilds the entries behind it in under a minute, but it does not cancel their old runs. Those keep going until they finish, and you pay for them.
  • So a flaky test is no longer just annoying: it ejects a pull request and costs a round of orphaned runs for everyone behind it. We fixed the two flakiest tests we knew about as we turned the queue on.

The first week

We compared pull requests of similar lifetime, because the ones opened since the queue started are, by definition, recent and short.

  • Fewer updates. Pull requests open under 4 hours that were updated with main went from 82 % to 27 %. Those open 4 to 24 hours, from 98 % to 40 %.
  • Fewer preview redeploys. Previews triggered by an update fell between 55 % and 62 % per pull request.
  • The queue costs something. Each pull request now pays one queue run, and it is always a full run of the whole suite.
  • The net, so far. For pull requests open under 4 hours, keeping up with main costs about the same as before (+6 %). For those open 4 to 24 hours it costs 22 % less. For longer ones, it is too early to say.
  • Flakiness shows. 9 % of queue entries failed and left the queue.
Minutes spent keeping a pull request up to date with main, before and after the queue Before the queue, keeping up meant branch updates. After, it means the remaining updates plus one queue run. Short PRs cost about the same; PRs open 4 to 24 hours spend 22 percent less. There is no data yet for PRs open over a day. Open < 4 h Before: updates: 100 100 After: updates: 22 After: queue run: 84 106 (+6 %) Open 4–24 h Before: updates: 190 190 After: updates: 62 After: queue run: 87 149 (−22 %) Open > 24 h Before: updates: 342 342 After: too early to measure Before: updates After: updates After: queue run
Index: update minutes of a PR open under 4 hours before the queue = 100. "After" covers pull requests opened and merged in the queue's first days, a small sample. The queue run costs about the same for every PR; the updates it replaces grow with how long the PR stays open.

That is a smaller headline than "we cut CI in half", and it is the honest one. The reason is visible in the chart: the queue run is a fixed cost per pull request, while the updates it replaces grow with how long the pull request stays open and how often main moves. For a pull request open under four hours, there is little to replace. For one open a day or more, there is a lot.

The queue removes the need, not the habit

The other reason the savings are still small: about a third of new pull requests were still updated with main. Some were real conflicts. The rest was reflex, people clicking a button they have clicked for years, and agents whose permissions still included gh pr update-branch.

So we changed the instructions too. The file our coding agents read before working in the repository now says it plainly:

### Merging — Do Not Update The Branch With Main
`main` merges through a merge queue: GitHub tests every queued PR
on top of the current `main` and merges exactly the commit it tested.

- Do not merge `main` into a PR branch, rebase it, click "Update branch"
  or run `gh pr update-branch` just to bring it up to date.
- Do update the branch for a real reason: a merge conflict, or a change
  on `main` the PR needs.
- To merge, use "Merge when ready" (`gh pr merge --auto`).

And I took gh pr update-branch off the list of commands my agents can run without asking. If updates only happen for real conflicts, keeping up with main becomes one queue run per pull request, whatever its lifetime: by our own numbers, about 16 % less than before for short pull requests, about half for those open 4 to 24 hours, and three quarters less for those open longer than a day.

If you are thinking about doing this

  • Measure the update tax first. Count the CI runs whose head commit is a merge from your main branch. If they are a large share of your minutes, a queue pays for itself; if your pull requests live an hour and merges are rare, it may not.
  • Fix concurrency before anything else. A queue run has no pull request number. Anything keyed on it breaks quietly.
  • Check every action that asks "what changed". Path filters, change detection and affected-project tools all need to understand the queue's event.
  • Fix your flaky tests. In a queue, a flaky test costs everyone behind the pull request, not just its author.
  • Change the instructions, not just the settings. If your agents were told to keep branches up to date, they will keep doing it, very diligently, after the reason is gone.

These are first-week numbers. We will come back to them once there are a few weeks of pull requests that lived their whole life under the queue, including the long ones, where most of the savings should be.