A number with no run behind it is a slogan. Every row here was produced on one machine by the command named beside it, and the ones that could not be measured cleanly say so rather than being rounded into shape.
Measured on 2026-08-02 at 0450f9e, 2026-08-02 at 0665561, 2026-08-02 at 43e43cf, 2026-08-02 at bdbc448, ? at ?. Re-run all of it with ./native/measure.sh; every value on this page is written by that run into site/measured.json and read back here, so a number and the run behind it cannot come apart.
Twelve open-source projects, each pinned to a commit, each installed and taken green before anything was measured. Flake is the share of clean runs that came back red with no patch applied at all, which is the number that decides whether a red run may be treated as a verdict. Locked is how many test files the judge hashes before a proposer is allowed to touch the tree.
Eight of the twelve produced no red run at all on a clean tree. 4 did, and those four are the reason the arbiter re-samples a red before it is allowed to become a verdict: on a suite that flakes, a single red throws away a correct fix somebody already paid for.
Four rounds of red-teaming this gate, 319 attempts, named 95 ways past it. One representative command each, judged by the binary this page describes: 37 refused, 58 allowed. They fall into four groups, and none of them is a surprise the code did not already suspect.
Enforce mode IS on, and these are its published limits. This line used to read "enforce mode is not on, and this is the reason" — true when it was written, false from the day the switch was turned on, and left standing on a page nobody re-read. A claim about the machine that outlives the machine is the same defect this project measures in other people's numbers. The escapes stay published beside the refusals, because a gate that promises a protection it cannot hold is worse than one that says what it stops. Reproduce with python3 redteam/redteam.py; it judges and never runs, with every destructive binary shadowed on PATH and the log of what they were asked to do asserted empty.
A quiet machine. Every run on this page shared its box with other work, so the wall-clock columns are contaminated and the pass or fail results are not. A second node version, to see whether the express flake follows the runtime. The date decay in date-fns, whose red rate climbs with the calendar. And the real test-file count in six of the twelve repositories, which leaves the question of how much of each suite the lock actually covers open in those six.
The sections of BENCHMARK.md and the verdict documents, one page each, with the command that reproduces them.
| measurement | in one line |
|---|---|
| gate latency | Gate latency: The gate runs on every coding-agent tool call (PreToolUse hook), on the critical path, so it has to be cheap. native/bench.py builds a scratch… |
| correctness | Correctness: Until fbd6f6e, native/postuse_test.sh piped identical hook JSON into both the Node gate and the native gate and asserted the same verdict (exit… |
| the ledger | The ledger: rabadon usage --days 30 aggregates the local spool (~/.rabadon/spool/). It excludes its own drills, demos and self-tests at emit — 3,495 events… |
| false rejects | False rejects: Latency says the gate is cheap. The ledger says how much it did. Neither answers the question a stranger actually asks — will it get in my… |
| a project rabadon has never seen | A project rabadon has never seen: §3b's 0.5 % is one operator, one machine, and rules that operator wrote. The question it cannot answer is the one that… |
| the refusals that cannot be undone | The refusals that cannot be undone: 524 refusals is the wrong number to quote and this section exists to say so. |
| not yet | Not yet: repairs held = 2. [building] This section read 0 for two weeks after it stopped being 0, which is the exact failure the rest of this page exists to… |
| live assist verdict | Decision: do not promote this candidate. The executive ran the live model experiment instead of passing that uncertainty to the implementing agent. |
| review a/b, 17 oct | A pre-registered A/B of rabadon's opt-in Stop review against plain Claude Code: evidence against at this scale |