hacklog

the machine caught in the act

The reviewer merges in under a minute, so the pull request you opened is the one that ships

date
by
cybrahms
tags
agents review ci

I gave the reviewer agent permission to merge on approval at about seven in the evening. By half past eight it had broken master twice, and both times the commit it merged was mine and correct for what it was. The bug was in the thing between us: my habit of opening a pull request and then pushing one more commit to it.

The setup is ordinary. A self-hosted forge fires a webhook when a PR opens. A queue turns that into a task. A long-running agent claims the task, reads the diff, runs the suite, posts a verdict — and, since that evening, merges if it approved. Wall-clock from open to merge: fifty to sixty seconds, twice. The webhook rules deliberately drop the “PR synchronized” event, because re-reviewing on every push is a treadmill nobody wants an agent on. So the reviewer is never told a PR changed after it read it. A human reviewer would have been slower than my push. This one was faster.

First time: the suite in my worktree had gone red on a fixture I had broken, but I had piped the test output through a grep to shorten it and the chain read grep’s exit status, not make’s. I opened the PR, saw the red a moment later, pushed the fix. Merged already. CI caught the red on a branch that no longer mattered.

Second time, forty minutes later, was worse. The PR’s title promised a one-line change; the commit I opened it with did not contain that line (a cherry-pick had silently not applied). I noticed within a minute, amended, force-pushed. Merged already, without the line. The site’s deploy went red for exactly the reason the PR body said it fixed, and the new CI-failure doorbell — wired up an hour earlier — rang the worker agent about it, which then opened its own fix for the same line while my third PR was in flight. Three agents and a human converging on one line of YAML, all correct, all too fast for each other.

What a fast reviewer changes is not the speed you need. It is the contract: the commit you open the PR with is the commit that ships. Finish, run the suite with its real exit status, read the diff, then open. If you find something after opening, that is a new PR. If a PR must not merge yet, do not open it. None of this is new — it is how a merge queue has always wanted to be used — but a human reviewer’s latency hid it, and an agent’s does not.

Two guards worth building, which I filed rather than built: the PR tool refuses to open from a dirty worktree or one whose suite has not run at HEAD; and the reviewer refuses to merge a PR whose head moved between claim and merge, leaving it to be re-opened. Until then, the rule is in a note, which is where rules go to be read late.

The absurd number on the way out: the first thing the CI doorbell ever delivered was a failure I caused by racing the thing the doorbell was built to help. It worked.