Jimmy Van Veen
By Jimmy Van Veen · September 28, 2026 · 7 min read

Auto-Merge for a Fleet of One

I copied Amplitude's risk-scored auto-merge onto my own repos. Two weeks later it had merged one pull request by itself, and the reasons were more useful than the feature.

Amplitude published a post about tripling pull request throughput in six months without hiring anyone. The headline is the 3x. The number I could not stop thinking about was cycle time, which fell from 5.2 hours to 44 minutes.

They got there in two moves. First they made CI fast. Frontend runs went from about 30 minutes to 3 or 4. Then they stopped putting a human in front of changes that did not need one. Every PR gets scored on risk, using size, scope, where in the codebase it lands, test coverage, and whether it touches a public API or a data model. Low risk merges on green. Medium and high route to a code owner.

I do not have code owners. I have me. I am the author, the reviewer, the approver, and the person who has to do this after a day job.

The bottleneck is me

Since I let coding agents write most of my side-project code, writing the code is no longer the slow part. The slow part is that every change queues behind one person, and that person also wrote it and already knows what is in it.

So the review I perform is mostly theatre. I open a PR I dictated an hour ago, skim a diff I already read, and click merge. Except sometimes I do not click merge for three days, because it is Tuesday and I am tired, and by Friday there are six of them stacked up and I have lost the thread on all of them.

Amplitude's answer does not depend on having a big org. Decide in advance which changes deserve a human, then let the rest go.

Three tiers

I collapsed their model to one question. Which changes need my eyes before they land, and which should never wait on me?

Low is every changed file matching a low-risk glob, meaning docs, tests, formatting and editor config, with a diff under 300 lines. It merges on green.

Medium is ordinary source code. It merges on green plus a clean verdict from the review bot. The bot posts one comment and its last line has to be exactly VERDICT: clean. Anything else and the PR sits.

High is anything under .github/, anything matching a per-repo protected glob, 600 or more changed lines, or more than three top-level directories touched. A human, always. The .github/ rule is not configurable by the caller, because a PR must not be able to argue itself down a tier by editing the classifier.

That rule fired on day one. The system labelled every PR that rolled it out to a repo risk:high, including the one adding it.

The workflows live in one public repo, JimmayVV/fleet-ci. A project calls them in a few lines.

1jobs:
2  ci:
3    uses: JimmayVV/fleet-ci/.github/workflows/node-ci.yml@v1
4    with:
5      package-manager: bun
6      playwright: true

Change a rule in one place, move the v1 tag, and every repo picks it up on its next run. New repos start from JimmayVV/fleet-template.

What broke on day one

Dependabot merged two PRs in twenty seconds

My poker clock repo is private on a personal account, and GitHub only allows rulesets on private repos on a paid plan. I wired up Dependabot auto-merge anyway and watched two dependency bumps open and close before CI had finished installing. PR #50 opened at 19:20:08 and merged at 19:20:28.

gh pr merge --auto does not mean "merge on green". It means "merge when nothing is blocking", and on a branch with no required checks nothing is ever blocking. Auto-merge becomes merge-now, with no error.

The fix is a guard every arming step runs first. It asks the API whether the branch has any rule requiring status checks, and refuses to arm if not. I would rather auto-merge not work at all than work by accident.

Dependabot does not write bun lockfiles

Every Dependabot PR on the bun repo failed with lockfile had changes, but lockfile is frozen. Dependabot edits package.json and leaves bun.lock alone. The two PRs that merged in twenty seconds carried that mismatch into main.

You cannot read secrets in a reusable workflow's with:

build-env: VITE_CONVEX_URL=${{ secrets.VITE_CONVEX_URL }} failed to parse. The secrets context is not available there, so the run showed up with a file path instead of a name. The fix was a real secret input on the reusable workflow.

The review bot will not review the PR that adds it

The Claude review job on the rollout PR went green in twelve seconds and did nothing. GitHub requires the workflow file to already exist on the default branch. That is a sensible guard, but a green check with no verdict behind it looks exactly like a passing review.

Two weeks later

I promised myself numbers, so here they are. Eight repos, September 14 to 28, 80 pull requests.

  • One PR merged the way I designed it to. A patch bump to image-size on this site got labelled low and merged 3.5 minutes after it opened, with no human involved.
  • Two more merged by bot. Those were the twenty-second accidents.
  • Twenty I merged by hand. That includes every medium-risk PR, and it turned out none of them could have merged any other way.
  • Forty-four were still open. Forty-one of those were from Dependabot.

The design was "decide in advance which changes need a human, then let the rest go." Almost nothing got let go, for four reasons.

Most of the fleet cannot auto-merge at all. Five of the eight repos are private on free plans. No rulesets, so the guard correctly refuses every time. Those repos get labels and reviews, and I merge everything.

The guard was right by accident. On those private repos the check for protection returns a 403. The line meant to turn a failure into zero, gh api ... 2>/dev/null || echo 0, kept GitHub's error text on stdout next to the zero. The comparison after it errored, and the script fell through to "not protected", which happened to be the correct answer. The one piece of this I would have called clever was working for the wrong reason.

The medium tier could never merge. The arming step only trusts a verdict from an allow-listed bot, and it checked the name with grep -qx "$AUTHOR". That treats the name as a regular expression, and in claude[bot] the [bot] is a character class that matches one letter. No bot name could ever match itself. Every clean review was read, logged as -> false, and ignored. I found it hours after publishing this post, when a clean review on this site's own PR armed nothing. The fix is one flag, grep -F (fleet-ci#12), and the next medium PR merged itself about a minute after its review.

Nineteen Dependabot PRs had no risk label. I had put the labelling inside the step that arms auto-merge, and that step only runs when the guard passes. On a private repo, patch bumps got no tier at all. Grouped and security updates, where Dependabot does not report an update type, fell through every branch. So the queue I was supposed to triage could not tell me which PRs mattered.

CI was the one clear win. The median run was 3.2 minutes on this site and 2.1 on the poker clock, both under the five-minute bar I set.

The fix, and the part that does not need one

The fix is in fleet-ci#10. Labelling is its own step now, and it runs before the guard. Patch and minor bumps are low. Everything else is high, with a warning that says why. Arming follows the label. The guard only accepts a bare integer.

That does not make private repos auto-merge. Nothing free does. It sorts their queue, so clearing it by hand takes minutes instead of an evening.

The twenty-two high-risk Dependabot PRs are the part that needs no fix. They are major version bumps, actions/checkout from 4 to 7 and the like. The system put them in front of me because a human should read them. That is the tiers working.

What the numbers decided

Counting the fleet showed me it was organised by accident. Products sat next to experiments on my personal account, all private, all unable to use the thing I had built for them.

So I moved them. Personal open source stays on my account. Products go to Redline Labs, a GitHub organisation that will pay GitHub's roughly $4 a month for private-repo rulesets when one of those products earns it. The automation is free. Turning it on for private code costs four dollars. I only learned that was the real trade by counting.

Copy it

Everything above lives in two public repos. PLAYBOOK.md is the policy in plain words. fleet-template is a repo with it already wired in.

For an existing repo it is four steps.

  1. Copy ci.yml, review.yml and risk.yml from the template's .github/workflows/, plus its dependabot.yml. Point ci.yml at your package manager.
  2. Put the paths that must always get a human into high-paths in risk.yml. Schema, auth, billing, deploy config.
  3. Apply the ruleset and turn on auto-merge in the repo settings. The check names in the ruleset have to match what your CI job reports, or nothing ever arms. My own template repo was missing its ruleset until the day I published this, and its first Dependabot PR sat there labelled low and never armed.
  4. Add a CLAUDE_CODE_OAUTH_TOKEN secret so the review bot can post its verdict.

Check three things before you trust it. A private repo on a free plan cannot arm at all, so decide whether that is fine before you wonder why nothing merges. Open one low-risk PR and confirm it waits for CI instead of merging on the spot. And if you use bun, Dependabot will break your lockfile on every bump.

Mergify, Aviator, CodeRabbit and Graphite all give a version of this away already. Mine is YAML you can read in an afternoon, which is the only reason to pick it. I will report again once the fix has had a month to run.