
Proof
Three small designed-suite confusion matrices, their exact binomial lower bounds, and a separate full census of the stored corpus. The 35/35 numbers are not the 2,098-file run. Point estimates alone would flatter small samples, so the bounds are shown next to them.
Confusion matrices
Recall with exact 95 percent lower bounds
All three suites observed 100 percent recall and zero hard false positives. The lower bounds tell the honest story about sample size: 90.0 percent for 35 fixture positives, 15.8 percent for 2 structural positives, and 2.5 percent for a single real positive. Only the fixture suite has enough positives to clear the 75 percent target with statistical confidence. The other two are regression evidence.
Census of the stored corpus
| Measure | Value | Source file |
|---|---|---|
| Fixture agreement | 100.0% | reports/labeled-latest.json |
| Structural regression recall | 100.0% | results/structural-latest.json |
| Selected real objective recall | 100.0% | results/selected-real-head-latest.json |
| Share of all real AI slop caught | unknown | not measurable from this corpus |
| Stored patch added line marker rate | 0.180% | results/professional-benchmark-latest.json |
| Executed cases across small designed suites | 55 | sum of the three suites above — not the full corpus |
| Full stored-patch run (separate) | 2,098 files · 113 blocked | docs/data/full-stored-patch-latest.json |
Two different results
The matrices and 55-case row above are designed fixture / structural / selected suites.
The full end-to-end run scored every stored prs/*.json file: 2,098 files, 113 blocked, 1,803 passed, 182 unscored.
Do not treat 35/35 or 38/38 as the full-corpus catch rate. Open the
measured full stored-patch run.
Executive verdict
AIV met the 75 percent point target on each small designed objective suite; the separate full stored-patch run blocked 113 of 2,098 files; broad real-world AI-slop recall remains unmeasured.
Next: why this matters for a project trying to avoid AI slop.