Four live runs, four different malformed-diff failures, and the obvious next move was the one that felt best: pick the most plausible-sounding explanation and go fix it. The most plausible explanation was that the retry loop wasn’t giving the model useful feedback. And it was a real bug. Fixing it did not fix the problem.
The actual problem: a correctly diagnosed bug that wasn’t the whole story
apply_patch_dry_run’s failure dict had no error key, only message/stderr/stdout. The retry-prompt code was reading .get("error", "Unknown apply_patch error") and silently falling back to that identical generic placeholder on every single apply failure, no matter what actually went wrong. Two real consequences followed: the model’s retry prompt never contained an actionable diagnostic, and the error-dedup check that stops the loop early saw the same placeholder signature after any two consecutive failures, cutting the loop short at two attempts even when each mistake was genuinely different.
That is a real bug, correctly root-caused, and it was fixed the same day: the retry prompt now carries the real git apply diagnostic, and the loop’s own dedup logic can finally tell different failures apart. Live-verified against real Ollama: a coding.bugfix run used its full four-iteration budget for the first time, hitting four genuinely distinct causes instead of deduping out after two. The mechanism demonstrably worked.
None of those runs converged. Not one cleanly-applying, test-passing patch. The mechanism was fixed. The model still struggled.
What got weird: two more plausible theories, both dead on arrival
The next session opened with a written investigation brief and two further hypotheses on the table. First: maybe the retry prompt was still, somehow, missing the corrective signal. Second: maybe the primary coder model was getting evicted mid-run in favor of a smaller fallback model that simply couldn’t hold the diff format.
Both got checked against stored evidence before either was accepted.
The retry-prompt hypothesis died first, and quickly: the harness had already been fixed the day before to interpolate both the failed patch and the real error into every iteration-≥1 prompt, and the stored prompt sizes proved it, one failing case grew from 2.5KB to 4.1KB between iterations, another from 2.9KB to 5.3KB, both carrying the real diagnostic. The model had the real error in front of it and repeated a byte-identical broken diff anyway. Whatever was wrong, it wasn’t a missing signal.
The eviction hypothesis died on a single query. model_load_events is a stored event log the harness already writes for every run, and reading it directly instead of inferring from behavior showed all four coder calls in the failing run were served by the same model the entire time, a smaller model only loaded afterward, for the unrelated summariser role. The theory had a clean, checkable answer sitting in a log file the whole time.
What worked: the discipline of checking before fixing
Three plausible-sounding theories in sequence (bad retry feedback, missing retry signal, mid-run eviction) got three checks against real stored evidence instead of a guess-and-patch cycle. The first theory was real but insufficient. The other two were flatly wrong. Every one of those checks was cheap: a byte count, a log read. None of them required writing a fix first to see if it worked.
What was left standing after three eliminations was the structural conclusion nobody wanted to reach first, because it’s the expensive one: the model could locate and describe the change correctly, every time, and still could not hand-author unified-diff syntax reliably, even with the exact failure modes spelled out in the prompt. Not a retry problem. Not a feedback problem. Not a model-selection problem. A syntax-authoring problem, sitting underneath three other theories that all sounded more fixable.
What next
Three theories down, and the real cause was now visible precisely because the cheap explanations had been ruled out on evidence rather than argued down in a meeting. The fix that followed would not be another prompt tweak. It would remove the requirement that the model write diff syntax at all.
What Next
Stop Teaching the Model diff Syntax
Next in the Local Coding Harness arc: with every cheap theory eliminated, the fix stops trying to teach the model to write diffs correctly and removes the requirement instead.