AIV Integrity Gate
Full corpus run · not the 50-case fixtures

Every stored patch file was scored.

This is the full end-to-end run on every prs/*.json file in this repo: 2,098 files, not the labeled 50-case fixtures and not a sample. The 15,575-row Airflow index has no diffs, so it was not executed.

Stored files scored
2,098
all corpus prs/*.json files
Blocked
113
gate findings, not “AI slop”
Passed
1,803
182 could not be scored
Airflow census \(context\)
15,575
Airflow metadata only, no diffs
Do not mix denominators. The 35/35 / 0/15 numbers below are a separate designed fixture suite. They are not the 2,098-file full run. The full-run receipt is full-stored-patch-latest.json.

1) Full stored-patch run VERIFIED

Every stored patch file under corpus/**/prs/ was executed with local aiv-cli 2.0. That includes Airflow e2e-proof (726), Airflow year folder (493), and six other projects (879). Total files 2,098 / unique PRs 2,080. Primary blocks: assertion-free tests 50, AI-generated markers 38, placeholder text 15, syntax 7, plus 3 other invariant rules.

Stored files
2,098
full corpus, not a sample
Blocked
113
gate findings
Passed
1,803
182 unscored
Airflow files
1,219
91 blocked

Every project table Every blocked PR Walkthrough · Raw: full-stored-patch-latest.json

2) Labeled 50-case fixtures VERIFIED · separate suite

Designed ground-truth fixtures only. Not the 2,098-file full run.

Labeled recall
—
35/35 expected-fail
False positives
—
0/15 expected-pass
Abandoned clean
—
separate abandoned sample
Hard-fail in sample
—
abandoned sample only

Loading fixture datasets…

Cohorts

    Precision: — · Cases: —

    Raw: labeled-latest.json · Live PRs: this repo

    Why this matters. Census catch-rates on unlabeled abandoned volume are not product claims. This layer answers: when we know the answer, does AIV get it right?

    3) Abandoned Airflow PRs → labels → AIV

    Same diffs, labeled and pushed through AIV.

      Among ai_disclosed: hard-caught — (disclosure alone is not a block reason).

      Example recreation: PR #72 recreates apache/airflow#73096 under label ai_disclosed — markers empty, local AIV hard=false. Open Actions on that PR to see the live gate run.

      Per-PR ledger

      Origin Label Hard Block Advisory Disclosed Bench Title

      Scoring note: hard column prefers aiv_head_full (head materialize + full config). Raw: abandoned-labeled-latest.json

      4) Confirmed ChatGPT gold set VERIFIED

      Tiny confirmed-disclosure cohort.

      Raw: airflow-chatgpt-true-latest.json

      4) Last 6 months and full year

      GitHub search totals for 22 Apr 2026–22 Oct 2026 and 22 Oct 2025–22 Oct 2026, next to the labeled sample stored in this repo. Search totals are not a census, and Generated-by on Flink and curl includes template text.

      Project PRs, 6 months PRs, 1 year Generated-by, 6 months Generated-by, 1 year Sample, 6 months Sample, 1 year
      Airflow8,55215,5755,1747,474305555
      Flink1,5842,4419861,0123251
      Kafka1,8133,18143553238
      curl1,8943,8533226571318
      Godot3,6687,58947793962
      Ghostty1,1922,49210273253
      COSMIC compositor242425672041

      The Airflow year is now a full index of 15,575 pull requests (4,329 disclosed, 7,265 merged, 2,182 closed unmerged, 1,526 tool mentions, 25 slop-language, 248 open). Counts: airflow-all-latest.json. Rows: corpus/airflow-all/prs.jsonl. Airflow issues stored: 1,100 in 6 months, 2,427 in the year. Dev-list messages: 1,965 and 4,377. Raw: windows-latest.json

      NOT CLAIMED policy

      Validate independently

      Everything on this page is backed by JSON in /docs/data. Live CI is the open PRs on this repository. Gate source and scripts live in aiv-integrity-gate.

      # From aiv-integrity-gate
      python benchmarks/airflow/scripts/generate-labeled-corpus.py
      python benchmarks/airflow/scripts/run-labeled-benchmark.py
      python benchmarks/airflow/scripts/benchmark-abandoned-labeled.py --limit 60 --inject --inject-clean 8
      python benchmarks/airflow/scripts/rescore-ai-signal-head.py
      # Compare outputs to docs/data/*.json on this Pages site