He hit 72 tokens/sec coding locally on a MacBook. The thread asked: why not just use Ollama?
A guide builds a fully local coding agent on an M1 Max: llama.cpp, Gemma 4, a speculative-decoding draft model for a 24% speedup. The top reply was a shrug — load Ollama, point your agent at it, skip all this.
The guide wires up an offline coding agent on a 64GB M1 Max: llama.cpp built with Metal, Gemma 4 in Q4, and a Multi-Token Prediction draft model that lifts generation from 58 to 72 tokens/sec.
The thread's sharpest critique wasn't the setup but the metric: every local-AI post benchmarks tokens per second, none benchmark answer quality. 'Quickly serving me slop doesn't make it more useful.'
A laptop coding agent with no cloud, no API bill
The motivation in the guide is mundane and relatable: the author's internet kept dropping, leaving him stranded mid-task. So he built a coding agent that runs entirely on the laptop. The stack: llama.cpp compiled with GGML_METAL=ON for Apple GPU acceleration, Gemma 4 as the main model in Q4 quantization (~16GB), a Q8 draft model for Multi-Token Prediction (speculative decoding), and the terminal agent Pi pointed at llama-server's OpenAI-compatible endpoint on localhost. Hardware: an M1 Max with 64GB unified memory.
The 24% that the draft model bought
The headline number is the speedup from adding the MTP draft model. On a short Python prompt (~128 tokens generated), the author measured:
| Setup | Speed |
|---|---|
| Gemma 4 baseline (llama.cpp Metal) | 58.2 tok/s |
| With MTP draft model | 72.2 tok/s |
| MLX-LM comparison | 38.1–45.8 tok/s |
That's the 24% gain, with prompt processing unchanged. He also notes Qwen3.6 35B is a better coding agent than Gemma 4 but slower (~55 tok/s), a quality-for-speed trade.
The thread's first reaction: why this much work?
The 99-comment thread was full of practitioners running their own local rigs, and the most common note was that the hard part of this guide is optional.
"or you can just load up ollama, have it load a local model and point claude or opencode at it… I'm not sure why he went through all the bother of llama.cpp" — a commenter who runs the simpler stack
Others pointed at LM Studio as the obvious GUI route, repeatedly. A reader poked a precise hole in the benchmark itself:
"Generating 128 tokens is probably not enough for good benchmark results. MTP speedup depends on how often the predicted tokens are accepted… short testing can give false positive speedups." — a commenter who pointed to llama.cpp's own sweep tool
The metric everyone benchmarks, and the one nobody does
The sharpest objection wasn't about tooling at all. It was about what these posts measure.
"My biggest pet peeve with all these articles on local AI is the only thing they talk about is tokens per second. No one mentions the quality of the answers. No one. I don't mind waiting a little longer if the quality is better. Quickly serving me slop doesn't make it more useful." — a reader tired of speed-only benchmarks
That reframes the whole genre: speed is easy to chart, usefulness isn't.
What the people actually doing it report
Underneath the methodology fight, the experienced voices were quietly enthusiastic about local-first work, slow as it is.
"Even though my local setup is a bit slow, it is a thrill to do real work completely locally." — mark_l_watson, author of 20 books on AI and a local-models practitioner
And a concrete data point on a heavier model than the guide's:
"I've been quite impressed with DeepSeek v4 Flash… it feels like a GPT-4 class model in terms of 'stored knowledge' but is better at long-horizon tool calling… Running on a 128GB MBP M4 Max, I'm getting ~24 t/s on generation and ~200 t/s on prefill… surprisingly useful as a 'machine orchestrator' for simple tasks." — a commenter running DeepSeek v4 Flash on a 128GB Mac
The consensus that emerges isn't about which tool wins. It's that local coding agents have crossed from toy to usable on a high-RAM Mac — and that the honest benchmark is whether the answer is good, not how fast the slop arrives.
Get breakdowns like this every day
The top Hacker News stories, unpacked — by email, Telegram, or RSS.
The one-line prompt that strips AI slop from a UI: 'make it look like a Qt app'
A developer who calls his own eyes tasteless found a trick: tell the model to style the page like a Qt desktop app, and the slop mostly evaporates. The thread explained why it works — and argued Qt is just an older slop.