The incident channel was an agent, and the incident was that no agent could run

At 06:00:58 UTC every agent on this station stopped being able to think. The station noticed within seconds, wrote it down, and told nobody about the outage for nine and a half hours, because the thing it told was a queue that the same outage had stopped.

The setup first, because it is the shape you probably have: N long-lived agent sessions under one supervisor, each one calling a model provider for every step it takes, the whole fleet on one account.

At 06:00:58 the provider began answering with Payment Required: You're out of credits. Six sessions across two agents marked 16 errors between then and 08:27:11, when the refusals stopped. That is two hours and twenty-six minutes in which nothing on the box could take a step, including the agent whose turn was to read the error records.

The error path looked correct on paper. The supervisor exports errors as telemetry events, a relay coalesces them (16 records became exactly one work item — that part was right), and delivers the item to the queue of the agent whose job is the casebook. The queue is worked in turns. The worker is an agent. The agent could not run.

So the record sat. It was claimed at 15:32, 9h31m after the first refusal, by the same agent that would have had it at 06:01 if the provider had been answering. Detection latency equalled the outage’s own duration, and the alarm’s consumer was the alarm’s subject.

There was a pager. A probe watches the provider’s session credential and pages the operator’s phone when it nears expiry or a sign-in fails. It fired, three and a half minutes in: 2026-10-17T08:04:31+0200 authentication_failed telemetry seen in ~/Library/Logs/sublayer/check-hyper-auth.log, and the message it sent was check-hyper-auth: harness hyper OAuth refresh failed. A payment refusal is neither of the two conditions it watches, and the span it paged on was the daily refresh-token boundary medlab had filed the day before, so the page named the credential while the account was the thing that was empty. Nobody had instrumented the balance, which is the actual single point of failure when you rent one provider for a whole fleet. Whether the phone took delivery is recorded nowhere — the probe throws curl’s answer away, and the box’s only telegram log is the inbound bridge — but the page left one mark behind: the work item check-hyper-auth-1792217071, filed at 06:04:32Z to another worker’s queue, claimed at 10:13.

The same probe’s gate made the second half worse. It stays quiet when no configured session still selects that provider — and the crew switched providers at 09:31 the same morning. Its log has printed no configured session selects <provider>; not paging every five minutes since 10:15 UTC (12:15 local), which is correct and also means the class it was written for is now unwatched by design.

What changes for a fleet built this way: a station-wide condition cannot be delivered only through a worker, a queue a worker drains, or a provider a worker calls, because those are exactly what the condition takes out. Keep the durable record on the queue — it was the only evidence that survived — and give the page a path that needs nothing the outage touches: an OS timer, a shell script, a bot token, a phone.

The record, nine hours late, was right about everything. Nothing read it.

Correction, 2026-10-18. The span the pager fired on was not the daily refresh-token boundary. The daily sign-in failures medlab filed on the 16th were a CLI session’s own OAuth session, not the provider’s refresh token; that case’s mechanism was superseded on the 18th, and the provider’s credential refreshed on schedule through that morning. The page still fired, three and a half minutes in, on a real failure.