Newer Claude models keep inventing tool fields that break Pi's editor
Armin Ronacher watched Opus 4.8 fill Pi's edit tool with fields that don't exist — requireUnique, oldText2, matchCase — while the actual text it wanted to change was perfect. The call failed anyway, and he thinks Anthropic's own training taught the model to do it.
Armin Ronacher, creator of Flask, found that Opus 4.8 and Sonnet 5 keep adding invented keys like requireUnique and oldText2 to Pi's edit tool. The oldText and newText payloads are byte-correct; the extra fields alone cause repeated failures.
His hypothesis: reinforcement learning inside Claude Code's forgiving harness rewarded malformed calls, so the model now fights any schema that isn't Anthropic's.
The thread debates whether that's a fixable bug or a walled garden.
Byte-perfect edits that still fail
Armin Ronacher, the creator of Flask, hit a strange failure in Pi, an open coding harness: newer Claude models (Opus 4.8, Sonnet 5) call its edit tool with invented fields stuffed into the nested edits[] array, keys like requireUnique, oldText2, and matchCase that the schema never defined. The oldText and newText payloads themselves are byte-correct. The extra keys alone get the call rejected, and the slip happens right after long escaped strings, where the model must choose between closing a brace or adding one more field.
His root cause is not random decay but a training artifact. Claude's newer models were trained heavily inside Claude Code, whose edit tool is flat and forgiving: it silently repairs malformed calls and drops unknown keys. If reinforcement learning runs in that harness, slightly broken calls still complete the task and still collect reward, so the model adapts hard to the one canonical shape. His warning, stated plainly: alternative schemas are not just unfamiliar, they may be implicitly punished by post-training that optimizes for a single forgiving tool ecology. A better model fights harder against anything off-distribution, and because Claude Code is closed, no one else can match its quirks on purpose.
The harness is now part of the model
Ronacher's failure reproduced fast in other people's setups. A software engineer in Christchurch who logs diff failures in his own harness saw the same pattern in patch generation.
"All models are terrible at generating line numbers for a proper diff, give up on them… Some models (Owl-alpha) must have been post-trained on Codex transcripts, because they occasionally push its V4A patch format into any diff tool available." — a New Zealand engineer running his own harness
A veteran developer saw it in piclaw, which rides the Pi runtime, and put a rate on it.
"and fable does the same, as far as I can tell, with one out of five or six edits failing. But I prefer OpenAI models for coding, so it wasn't a real problem." — a developer who hit the same bug in piclaw
Not everyone thinks the fix is deep. One engineer argued the whole thing dissolves with a good error message.
"I did a revolutionary thing and made the error output print helpful guidance on how to correctly call the tool. The agent tries again and always gets it right." — a developer who patched it with better errors
A more structural fix surfaced too: constrain the sampler so illegal fields cannot be emitted at all.
"Surprised models still output tools as text when for ages we've been able to constrain the output at the inference engine level." — a commenter pointing to grammar-constrained decoding
The reframing one reader landed on outlasts the bug report: intelligence now lives in the harness plus the model, not the model alone. Whoever's harness did the training gets to set everyone else's defaults.
Get breakdowns like this every day
The top Hacker News stories, unpacked — by email, Telegram, or RSS.
Webb keeps finding black holes too big, too early to exist
A billion-solar-mass black hole sits 650 million years after the Big Bang, feeding at 40 times the speed limit physics allows. Webb keeps finding more like it.