80 tokens/sec on two mismatched GPUs — and a thread asking if self-hosting still makes sense
A 16GB Blackwell RTX 5080 and a refurbished 24GB Ampere RTX 3090, split across one consumer board, run Qwen 3.6 27B at Q8 and over 80 tokens a second. It's a clean recipe for mixing GPU generations.
The trick is llama.cpp with speculative decoding and a tuned tensor split across two architectures Nvidia never meant to pair. The catch a reader measured: about 700W under load, enough to heat the room.
The thread turned into a referendum. One reader pays roughly $3 per million tokens in the cloud, unquantized, for the same model; another mourns that the run-your-own-hardware ethos is 'basically dead.'
The build: two GPU generations, one model
The writeup documents a deliberately mismatched rig: a 16GB RTX 5080 (Blackwell) beside a refurbished 24GB RTX 3090 (Ampere), on an Asus Prime X570-Pro splitting PCIe 8x/8x. Running Qwen 3.6 27B at Q8 through llama.cpp with CUDA, it reaches 80-plus tokens per second — up to 90-plus — at a 230K context with a quantized KV-cache. The performance comes from multi-token-prediction speculative decoding plus ngram drafting and a hand-tuned tensor split (-ts 2,3) across two architectures that were never designed to share a workload. The takeaway is practical: you don't need a matched pair of cards to run a large quantized model at usable speed.
The thread sharpened the author's own settings. A very-high-karma reader corrected the sampler and speculative-decoding parameters in detail:
"The recommended values for Qwen 3.6 in thinking mode is
--temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.00… Also, the recommended Qwen MTP settings are--spec-type draft-mtp --spec-draft-n-max 2. 3 is not good on Nvidia hardware." — a reader with detailed Qwen tuning knowledge
The number the post under-sells: 700W
A reader with a wall meter supplied the operating cost the build glosses over:
"it now draws 700W on full load. Just a single 5080 is enough to measurably heat up a room when loaded (320W draw at the wall on mine)… you better have a good PSU as well as checking your power plugs themselves." — a reader measuring draw at the wall
Self-host or rent? The thread can't agree
That power figure fed the real argument, which wasn't about tokens per second at all. The cloud-economics case was direct:
"I pay ~3$ per 1M/tokens for that model on Openrouter, and it's not even quantized. A refurbished 3090 and a 5080 will set you back well over 2k, not to mention the electricity…" — a high-karma reader doing the cloud math
But someone running the identical setup made the case that local isn't about price — it's about how the model fails:
"I started to prefer my local Qwen3.6 35B A3B and pi agent over Claude Code… the way Qwen fails is much more straightforward… it just goes into a tool calling doom loop that the pi harness can catch, whereas Claude attempts ever more convoluted and creative things just making more and more mess." — a reader running the same rig
And one voice mourned the culture shift underneath the whole debate:
"Open source, run-your-own, pay for nothing… ethos seems basically dead. I guess I'm getting old." — a reader who owns two 16GB cards
The recipe works; the question is why you'd run it
As a technical artifact the build is a small success — proof that heterogeneous cards and good speculative decoding get you frontier-ish throughput on hardware you own. As an economic proposition it's genuinely contested: $2,000 of silicon and a 700W draw against $3-per-million cloud tokens is a bet on privacy, predictability, and failure modes you can debug, not on saving money. The thread's most honest read is that self-hosting in 2026 is a values choice wearing a benchmark.
Get breakdowns like this every day
The top Hacker News stories, unpacked — by email, Telegram, or RSS.
An AI one-shotted a herding game in 45 minutes. The thread found it on the Game Boy Advance
Koen van Gilst had Anthropic's Fable model generate a complete sheep-herding browser game in a single pass — 2,319 lines, no dependencies, for 45 minutes and €20 of tokens. The headline calls it the model too dangerous to release.