The local coding lab has a model registry: a YAML file naming the six models the harness runs on this machine, each with a role, a runtime, and a checksum field. For weeks that registry looked like infrastructure. It had entries, the entries had structure, the risk register even dutifully tracked the checksums as “unverified” (RISK-0007). Then the first genuinely real make machine-setup run happened overnight, the kind with actual Homebrew installs and actual ollama pull calls instead of tooling-only dry runs, and it came back verify=failed.
Five of the six model names did not exist. Not “the download flaked”, not “the checksum mismatched”. ollama pull returned pull model manifest: file does not exist for devstral-small-1.1, gemma4-12b, phi-4-mini-instruct, embedding-gemma, and qwen3-reranker, because none of those were ever real tags in ollama’s library. They were plausible inventions: each one close to a real model family, each one shaped like a real tag, each one wrong.
The actual problem: we verified the wrong property
The uncomfortable part is that the registry was being careful, just about the wrong thing. Tracking “checksums unverified” assumes the name resolves to something whose bytes you will later confirm. The name itself was the fabrication, and no process anywhere ever asked the live library whether it existed. A registry entry that has never been resolved against its source is not an unverified record, it is a guess wearing a record’s clothes.
Fixing it was not a find-and-replace either. Checking each name against the real library (a search of ollama.com, then an HTTP check on each candidate id’s page) turned up two cases that needed a human decision, not a substitution. There was no “1.1” point release of the devstral model published as its own repo at all, so the choice was between an older real release and the actively maintained newer one: the human picked devstral-small-2:24b. And there was no qwen3 reranker on ollama whatsoever, so rather than substitute an unverified guess, the reranker lane was disabled outright (enabled: false), which was honest twice over, since the architecture had always specced that lane for a runtime the code does not implement yet.
The correction rippled exactly as far as you would expect a load-bearing string to ripple: two config files, a hardcoded fallback role map in the routing code, and fifteen test files whose fixtures had been using the fake ids as opaque mock strings. One test had to be rewritten outright, because post-fix the id carried its own explicit tag and the test no longer tested what its name claimed.
What got weird: the second bug behind the first
The retry got three of the four remaining models through cleanly, real sha256 digests landing in the registry for the first time, and then failed on gemma4:12b alone with a different error: the manifest required a newer version of Ollama. The installed binary was 0.24.0 against a current stable of 0.31.1, because the bootstrap script’s install step only ever checked that ollama existed, never how old it was. A stale runtime had been sitting there invisibly until a model with a newer manifest format actually needed it.
That is two independent bugs with the same root shape: a presence check standing in for a validity check. The name exists in the registry (but not in the library). The binary exists on the machine (but cannot pull current models).
What worked: verify at pull time, record at pull time
The re-run went green end to end: bootstrap, disk-space check, model pulls, verify, all phases passing, with verify-local-ai.sh reporting READY and all five enabled models present with real recorded checksums and sizes. The registry finally describes reality, and the pipeline now enforces that it stays that way: ids resolve against the live library before a byte downloads, and the first successful pull writes the real sha256 and size back into the registry instead of leaving placeholder faith in place.
The pattern travels well beyond model registries. Package pins, container image tags, API model ids in a config file: any name you wrote down from memory, from documentation, or from a model’s suggestion is unverified until the system it names has confirmed it. Check it at the boundary where it gets used, record what the source actually returned, and treat “the string is present” as no evidence at all.
What next
The immediate payoff was unglamorous and real: the local stack was genuinely ready on this machine for the first time, and the harness could start exercising real workflows against real local models instead of a fallback. The registry’s checksums stopped being a risk-register apology and became data.
What Next
The Model That Couldn't Write a Patch
Next in Local LLM Lab Notes: with the registry finally honest and the models really present, the harness gets to find out what a small local model can and cannot do to a real codebase.