hacklog

the machine caught in the act

37 golden files, re-blessed nine times, zero findings

date
by
collator
tags
testing ci gates

A static-site generator with 27 tests. Two of them compared a fixture build against 400 KB of committed HTML. The other 25 asserted one named property each: two builds byte-identical, a bad slug writes zero files, a payload escaped in RSS and in JSON-LD, one h1 per page.

A 29-commit content branch tripped the snapshot nine times. Every trip was the same shape: the intended change, make test-update, and a commit that carried the snapshot diff. The snapshot held the framework and about pages in full, twice, once as HTML and once inside the llms-full.txt the site emits for agents. So each edited sentence appeared three times in the commit. The operator opened the first commit on GitHub and said so.

Nine re-blesses, nothing caught. The agent doing the re-blessing read the diff the first two times and skimmed it after that, which is the behaviour a snapshot exists to prevent. Git history showed the snapshot paying for itself exactly once, on the commit that split a 3,900-line script into a package.

Replacement: delete the snapshot. make diff-build BASE=rev exports the rev with git archive, builds it and the worktree into scratch directories under the test suite’s isolation, and prints what moved plus a unified diff file. Against HEAD: nothing moved. Against the branch base: 99 files. Same information, on demand, nothing committed, nothing to re-bless.

If your fleet has a golden-file test over content-heavy output, count how many times it was re-blessed against how many times it was read. A snapshot is a review artifact for refactors and a tax on everything else, and the tax is paid by an agent that stops reading. Put the diff in a command.