A new evidence-correlation product went through three adversarial review rounds in one build session (same-model, then a fresh independent pass, then a third round), and each one found the previous round’s fixes incomplete. Two Critical and six High findings in round one. Three in round two. Two more in round three. The count never reached zero, and it did not need to for the pattern to be worth naming: the fixes that lasted changed a rule; the ones that came back had only changed a line.
The finding that kept returning
Correlation, the logic that ties a scanned event back to the evidence that produced it, produced a new defect in every single review round: the one class not demonstrably closed by any of the three passes. A fix in round one closed a specific correlation gap; round two found the same underlying assumption broken in a different code path; round three found it again, one layer deeper. Each individual fix was correct for the line it touched. None of them changed the assumption that kept producing new instances of the same bug in new places.
The clearest illustration of what that assumption cost showed up as the most serious defect of the whole session: a replayed run reported “all evaluated controls behaved as declared” over a batch that actually contained a masking regression and an ungoverned invocation. Confident false reassurance: from a diagnostic tool whose entire purpose is catching exactly that failure mode elsewhere. It survived review because it was a side effect of idempotent replay, a property an earlier commit had cited as evidence the system worked correctly. The property was real. The inference drawn from it was not.
Rule versus line, made concrete
A line-fix changes the specific code that produced this instance of the bug. A rule-fix changes the invariant that would otherwise let the next instance exist. The correlation defects across all three rounds were, in retrospect, line-fixes: each one closed the specific gap the reviewer happened to find, and each closure left the generating assumption (that a replay run’s structural properties are sufficient evidence of correctness) untouched. The pattern only became visible in review, not in any single fix; no round could see that its own correction was one instance of a wider miss, because seeing that required the vantage point of the round after.
Why three rounds, and why it stopped there
The team’s own accounting of the session records it plainly: every review round found the previous round’s fixes incomplete. That is not a report of process failure. It is what an honest multi-round review looks like when the underlying defect class is genuinely subtle: the alternative, a single round that reports a clean bill of health, would have shipped the confident-false-reassurance bug into production with a passing review attached to it. Three rounds surfaced the pattern; a fourth might have surfaced more of it. The decision to stop was a judgment call about diminishing returns, not a claim the class was closed.
The part worth stealing
When a review round reports a fix, ask one more question before closing it: did this change the rule that generated the bug, or just the line the bug happened to be on this time. If you can’t answer that with confidence, assume line, and expect the next round to find a sibling.
What Next
Doubt Before the Code Exists
Doubt-driven-development moved earlier: applying the same adversarial scrutiny to an intent declaration before a line of code exists to review.