Four M5 Ultra Mac Studios linked over Thunderbolt 5 run an 816B-parameter model at 36 tokens a second
Muse Spark Open needs about 493GB for its 4-bit weights alone, so no single Mac can hold it. Apple's answer is to split it across several.
Apple's MLX team tested tensor parallelism over RDMA on Thunderbolt 5: four machines decode at 36.73 tokens/s at 8,192 tokens of context, against 15.74 with two-machine pipelining. A single M5 Ultra is about 1.6x faster at decoding than the M3 Ultra.
Splitting a model that does not fit
Apple's research post benchmarks Muse Spark Open, an 816B-parameter mixture-of-experts model with 42B active, in 4-bit NVFP4, plus two dense models: Mistral Medium 3.5 (128B, FP8) and Qwen 3.8 (27B, BF16). Hardware is Mac Studio with M5 Ultra (1.2 TB/s memory bandwidth, up to 512GB) against an M3 Ultra (819 GB/s), connected by RDMA over Thunderbolt 5 with macOS 26.2 and MLX v0.30.1 or later.
Two schemes were compared: pipeline parallelism, which splits by layer depth, and tensor parallelism, which splits each layer's width and needs low-latency links. On four M5 Ultras with tensor parallelism, Muse Spark Open decodes at 36.73 tokens/s at 8,192 tokens, versus 15.74 tokens/s with two-machine pipelining; Qwen 3.8 reaches 50.89 and Mistral Medium 3.5 reaches 23.37. The M5 Ultra's neural accelerators cut time to first token up to 4.6x versus M3 Ultra; Qwen 3.8 drops from 18.2s to 4.1s. At 65,536 tokens, two-machine pipelining prefills up to about 1.8x faster than one machine. Peak power is about 200W for the M3 Ultra and 300W for the M5 Ultra.
Hardware price and context length decide who can use this
A developer with 35 years in the field, ten of them at IDEO called it "a bit academic": the 256GB Mac Studio starts at $9,499, the 512GB price is not announced, and "when you start talking about four of those puppies, it's substantial." Against that, a commenter who benchmarked DeepSeek 4.1 Flash and GLM 5.3 Flash on M5-class machines reported a single 256GiB M5 Ultra running the first at about 45 tokens/s, and reported checking quality on MathArena Apex and GPQA Diamond against official numbers.
The post's tables also do not label context size per row, so treat the headline 36.73 as the 8,192-token case. For a buyer, the comparison that matters is one big Mac running a smaller model well versus four Macs running Muse Spark Open at a context that short.
Get breakdowns like this every day
The top Hacker News stories, unpacked — by email, Telegram, or RSS.
PS5LM turns a jailbroken PlayStation 5 into an OpenAI-compatible LLM server
Llama.cpp runs on the console's GPU through Vulkan, and every device on the home network can talk to it.