Two Bots Argued on a Pull Request for 21 Hours

On the night of July 31, one of our scheduled AI routines opened a pull request, and our AI code reviewer left comments on it. The routine fixed what the reviewer flagged and pushed. The push triggered a new review. The new review found something else. They kept going for about 21 hours, through more than 49 review runs, and the bill showed up as roughly $500 in automatic credit reloads on our API account. Neither bot was broken. Each one did its job, and that’s why it took a day to stop.

The setup

Our company brain runs a handful of headless routines on a schedule. These are Claude Code sessions with no human in the chair. One of them, the nightly repo routine, does a repo-only slice of our end-of-day process: it checks whether summary docs have drifted from their sources, refreshes stale banners, writes a short report, and opens a PR with the changes.

Our main branch is protected. The one required check is an automated review: a GitHub Action that runs Claude against every PR and posts comments. Routines that produce mechanical output (index regeneration, banners, reports) are allowed to auto-merge once that review passes.

So the intended flow was simple. The routine opens a PR, the reviewer looks at it, the PR merges, everyone goes to sleep.

Two settings made it a loop. The reviewer ran on every push to a PR, which is a sensible default for PRs written by people. And nothing in the routine’s prompt told it to ignore review feedback, so it did what a helpful model does with a comment: it fixed it. That’s also sensible, right up until the reviewer is another model that will always find one more thing to say.

How it played out

A model reviewing a diff almost never says “this is perfect.” It’ll suggest a clearer phrase, flag a date format, question a heading. A model responding to that review almost never says “I disagree, leaving it.” It makes the change and pushes.

Every push re-fired the review. Every review cost a metered Sonnet run. The routine session was still alive and still dutifully addressing comments. Nothing in either system had a notion of “we’ve been at this for six hours, maybe stop.”

An unrelated rule ended it. Each routine closes its own still-open PRs from previous nights before opening a new one, so PRs don’t pile up. The next night’s run started, saw the old PR still open, and closed it. That’s the only reason it was 21 hours and not a week.

The part that stung more than the $500: we had a rule that routines must never call a paid model API. The routine obeyed it. It never touched an API key. The spend happened one hop away, in the reviewer it kept waking up. A cost rule that only looks at what a process calls directly misses every cost that process can cause downstream.

The fixes, on both sides

We fixed it from the reviewer’s side and the routine’s side, on purpose. Either fix alone would have stopped this incident. Having both means the next variation has to get past two guards.

The reviewer stopped reacting to routine pushes. PRs from routine/* branches got reviewed once when opened, never on later pushes. Dependabot bumps and PRs with 30 or more commits were skipped entirely. A per-PR concurrency group collapses a burst of pushes into a single review run instead of one per push. One detail we had to check: when the review job skips itself, GitHub reports it as “skipped,” and that still satisfies the required check, so auto-merge keeps working.

The routine stopped arguing. A new rule went into the output convention every routine follows: once the PR is open, the run is complete. At most one correction push, and only if the reviewer found a genuine factual error. All further review comments are ignored. The rule includes the incident date and the 49 runs, so anyone who later wonders why it’s there can see the reason without digging through git history.

Only mechanical routines auto-merge. Routines whose output is derived and idempotent (regenerated indexes, banners, health reports) can merge on a passing review. Anything involving judgment, like a content sync or a security scan, leaves the PR open for a person.

Spend became visible. Our weekly review now has a “metered spend, month to date” line pulled from the API console. If the admin key that reads it isn’t configured, the line prints “not measurable” instead of disappearing. A missing line looks like zero spend. An explicit “not measurable” looks like a gap someone should fix.

The follow-up review found more

A few days later we went back over the review workflow with fresh eyes, and the incident turned out to be the least embarrassing thing in it.

First, nobody was reading the one review that routine PRs still got on open. So we dropped reviews on machine-authored PRs entirely: anything from a bot account, plus our routine branches, which push under a human account and would slip past a simple bot check.

Second, the reviewer had been pinned to a specific model since mid-March. That pin never took effect. The action doesn’t have a model input at all. It had been printing an “Unexpected input” warning on every single run for almost five months, and every review in that time ran on whatever the action’s default was. We moved the pin to the argument passthrough the action actually documents.

Third, the workflow granted write access to the repository at the top level, which applied to both jobs. The review job only needs to post comments, but because of that grant, it could have pushed code. Each job now declares its own minimum permissions.

None of that caused the loop. All of it had been sitting there quietly, and we only found it because the loop made us look.

Cost guards one hop out

The same month, we found our own test suite had been billing a metered API key on every run. Six tests called an LLM scoring function without mocking it, and the test process inherited the real key from the environment. The fix was a tripwire that strips model API keys from every test that isn’t explicitly marked as needing the network, plus two tests that make sure the tripwire itself can’t quietly stop working. The suite got about a minute faster, and no test turned out to need a live key.

In September we rewrote the “never call a paid model API” rule completely. The new version allows paid APIs when they do the job better, with conditions: every call site gets registered, it falls back to the session if the API fails, it stays advisory at any human approval step, it gets shadow-tested against our own past decisions first, and it reports its cost. The old rule was a ban that looked strict and missed the actual spend. The new one is permissive and asks every call to show up somewhere a person will see it.

What I’d take from this

When two automated systems talk to each other, assume one of them will never say “done.” Models are agreeable, and two agreeable models in a loop will happily agree with each other until something external stops them. Put a hard cap on the number of turns, put it in more than one place, and make sure the cap doesn’t depend on either model deciding to stop.

And watch the costs your automation causes downstream as well as the ones it incurs directly. Our routine had a spotless record on API calls and still ran up a $500 night.

This is one piece of a bigger system. The pillar post covers the rest of it. If you want something like this running for your team, with the guardrails already learned the hard way, I build them in 14 days.