A vLLM setting took Kimi K2.6 tool calls from 92% valid to 100%
Ubicloud first forked vLLM to fix malformed tool calls, then found upstream already had the cure behind a default that had changed.
Ubicloud measured 92% schema accuracy on Kimi K2.6 tool calls with vLLM 0.28, about 2 failures in 25. Turning on constrained decoding explicitly brought it to 100% on the K2VV ToolCall benchmark.
The thread's pushback: the fix was a configuration, not a patch.
The bug was a changed default
Ubicloud's write-up: Kimi K2.6 produced tool calls that did not match their schemas. Ubicloud forked vLLM to fix it, then switched to upstream and watched accuracy fall to about 91%, because vLLM had changed the default so constrained decoding is only applied if a request enables it. In vLLM 0.31 it is a server-side flag using XGrammar structural tags, which block invalid tokens at each step. Ubicloud enabled it for Kimi and DeepSeek models.
"They just needed to configure it correctly"
"Tool calling is fraught with peril, especially because there's nothing that necessarily says that a models tools are fixed in place for the life of its run." — a contributor who worked on the vLLM feature
"Schema-constrained decoding changes the distribution the LLM will output." — a commenter
He wanted quality numbers and asked how simple retry-on-failure would compare: at 23/25 success, about 9% extra tokens on the tool call itself. If you self-host open-weight models behind an agent, test your own tool schemas against the server flag before trusting a default.
Get breakdowns like this every day
The top Hacker News stories, unpacked — by email, Telegram, or RSS.
CSS-Tricks is independent again, and the thread asked who still reads CSS tutorials
DigitalOcean bought the site from founder Chris Coyier in 2022. On October 26 it went to Geoff Graham, who has worked on it for 12 years.