Local LLM Lab Notes

Public Posts

• Local LLMs and the Offline Factory

  The question stopped being can this Mac run a model, and became can the factory decide which worker does which job, with evidence.

• Five of Six Model Names Didn't Exist

  The first real overnight machine-setup run failed on model names that looked completely plausible and were completely invented. The registry had tracked their checksums as unverified for weeks. Nobody had ever checked the names.

• The Indexer Nobody Called

  The symbol indexer was built in week one. It had tests, structure, and a job. Weeks later, wiring up the graph layer, I found the truth in passing: no production code had ever called it. The repo map had been silently empty on every real run.

• The 3B Model Writes the Brief

  The small local model keeps failing at the actual work, so the obvious conclusion is that it's useless. The gateway found the job it's genuinely good at: writing the structured brief a frontier model executes. The intern doesn't do the surgery. The intern preps the theatre.

• The Model That Couldn't Write a Patch

  A local model can diagnose a bug correctly and still fail to ship the fix. Four live runs, two models, two scenarios, three genuinely different ways of getting the diff syntax wrong.

  Part 1 of 7 in The Local Coding Harness
• Falsify Your Favorite Hypothesis First

  Every plausible explanation for the malformed-diff pattern got tested against stored evidence instead of assumed, and every one of them was insufficient or flatly wrong, before the real cause finally cleared.

  Part 2 of 7 in The Local Coding Harness
• Stop Teaching the Model diff Syntax

  After three eliminated hypotheses, the fix wasn't a better prompt. It was removing the requirement that the model author diff syntax at all, and the first live attempt afterward finally produced a clean-applying patch.

  Part 3 of 7 in The Local Coding Harness
• The Bug Behind the Bug

  Fixing the diff-authoring problem let a patch survive for the first time in the workflow's history, and immediately tripped a second bug that had been invisible since day one.

  Part 4 of 7 in The Local Coding Harness
• A Review Panel on One Model Is One Reviewer in Four Hats

  A hardware ceiling forced a four-persona review panel onto one resident model. Measuring it honestly, against both a bugs-seeded fixture and a genuinely clean one, found a real instruction-following ceiling the panel couldn't tighten past.

  Part 5 of 7 in The Local Coding Harness
• Make the Reviewer Quote Its Evidence

  The same trust shift that fixed diff generation fixed the review panel too: a second-pass verifier that must quote an exact line from the diff, mechanically checked, not trusted on its word.

  Part 6 of 7 in The Local Coding Harness
• Two Labs, No Cloud, One Fix

  A cross-repo path-conflation fix turned a harness that only worked on itself into one that could fix a genuinely foreign repository. The same day, a sibling lab's fourth attempt at a real bug closed, entirely through local models.

  Part 7 of 7 in The Local Coding Harness

Draft Queue

No draft placeholders assigned to this series yet.