Three rounds of prompt tightening had brought the review panel’s false-approval rate from 3/4 to a stable 1/3 and then stopped moving. The remaining false approval broke the panel’s own explicit, unconditional hedge-word rule. A fourth prompt variant was unlikely to fix a model that wouldn’t reliably obey the third one. The pattern that had already worked once, on diff generation, pointed to the real move: stop asking the model to self-certify, and check its claim instead of trusting it.

The design: three choices made before writing any code

The fix that shipped was a second-pass verifier, inserted between the panel’s clustering step and its final synthesis, with three deliberate constraints baked in from the start rather than discovered by trial and error.

Downgrade-only, never drop. An unsupported critical or important finding gets downgraded to minor, which the precision check doesn’t count as a false approval, but its underlying text is never deleted. This means the verifier structurally cannot make recall worse: recall is matched on finding text, not priority, so anything the verifier downgrades is still there for a human or a downstream check to see if it matters.

A deterministic evidence backstop. The verifier’s own top-flagged risk, named explicitly in the phase brief before it was built, was that a same-model verifier might just inherit the exact instruction-following ceiling that produced the false approval in the first place. So “confirmed” is never taken on the model’s word alone: the verifier must quote one exact changed line from the diff, and a mechanical check, whitespace-normalized, with a minimum length floor, confirms that quote actually appears in the diff’s real added or removed lines. A missing or unverifiable quote downgrades the finding in code, no matter what the model’s verdict text claims.

Every flagged item gets challenged. Four personas converging on the same finding is weaker evidence than it sounds, because they share one model’s blind spots, confirmed the same day the verifier shipped, when all four personas converged on a false claim about a different module’s untracked-file handling during an unrelated local review. Convergence alone doesn’t earn a pass.

What worked: zero false approvals, on the fixture that mattered most

Live-verified against real Ollama on the original precision control fixture: false approvals dropped from 1/3 to zero. The verifier downgraded exactly the hedge-worded finding that had survived every previous prompt tightening pass, for exactly the right reason, its evidence quote either didn’t exist or didn’t hold up, and the mechanical check caught it where the model’s own stated confidence hadn’t.

What got weird: a second control fixture found the backstop’s real edge

Rather than let a single fixture carry the whole zero-false-approvals claim, a second, independently built precision fixture (clean billing-domain code, verified against a reconstructed baseline) was run through the same verifier. It produced the session’s honest surprise: one false approval got through. The verifier had confirmed a consistently-applied exception-narrowing change as a genuine regression.

The evidence backstop had done exactly what it was built to do (the quoted line was real, the diff genuinely did remove a broad exception catch), and that was precisely the gap. A quote-checking mechanism can confirm “this text really appears in the diff.” It cannot arbitrate “and that change is a defect, not an improvement.” The distinguishing signal turned out to be in-diff consistency: the earlier fixture had seeded the same pattern as a real bug by narrowing one sibling function while leaving its twin untouched; this clean fixture narrowed both, uniformly. One targeted verifier-prompt addition encoded exactly that distinction: a uniformly-applied change reads as deliberate, not broken.

What worked, fully: the promotion bar, met

With both fixtures re-run at the final verifier version: precision-001 held at zero false approvals across two consecutive confirmation runs, precision-002 matched at zero, and both recall fixtures held their existing scores (1.0 and 0.875) unchanged; the verifier’s downgrade-only design meant it could not have moved them. For the first time, every number the design doc’s promotion bar named was met on every fixture at once.

What the session deliberately did not do: flip the panel’s promotion status in the harness config itself. “The bar is met on current fixtures today” and “this panel should become the primary reviewer” are different claims, and the second one still carries the panel’s structural blind spot, one resident model’s whole-file-context miss, the class of issue only visible by reading beyond the diff, which a diff-scoped verifier cannot touch. That decision was left explicitly to a human.

What Next

Two Labs, No Cloud, One Fix

Next in the Local Coding Harness arc: the human makes the promotion call the verifier deliberately left open, and the fixed loop gets its first real test outside a scratch fixture, a genuine cross-repo bug, fixed by a sibling lab, with zero cloud calls.