Google's DiffusionGemma writes a paragraph at once — 4x faster, and a bit dumber

A printing press instead of a typewriter

Google's announcement describes a genuinely different generation method. Where an autoregressive model emits one token at a time, DiffusionGemma "drafts an entire 256-token paragraph simultaneously," then "makes multiple passes, locking in correct tokens and using them as context clues to refine the rest." It's an Apache-2.0, 26B Mixture-of-Experts model that activates only 3.8B parameters at inference and fits "comfortably within 18GB VRAM … when quantized."

The speed numbers, and the honest caveat:

Google states plainly that "DiffusionGemma's overall output quality is lower than standard Gemma 4," recommending regular Gemma 4 when quality is critical. The most striking demo was a fine-tune that solves Sudoku — a task "where each token depends on future tokens," exactly the case where committing left-to-right hurts and bidirectional refinement helps.

The voice that reframed it: "stupid fast" beats "smart"

The most valuable comment came from someone who'd already lived with a diffusion model in a coding loop, and described a qualitative shift rather than a benchmark:

"My unexpected favorite model to use was Mercury (a diffusion model). Not because it was 'smart' but because it was stupid fast. It was more of a pair-programming experience instead of the SOTA agentic experience of prompting and waiting. … It felt less of a slot machine where you prompt, wait, and hope." — vineyardmike

Where diffusion actually wins: the edge, not the datacenter

A founder/CTO corrected the most common misread — that diffusion is just generically faster. He located the real advantage precisely:

"An LLM's decoder computes tokens one-at-a-time because attention has to account for each previous token. The existing LLM decoders scale well when you … batch many inferences together. Diffusion of limited benefit there. On edge you have a different problem: your inference accelerator is starved while sloshing GB of weights back and forth from RAM. … Diffusion can compute tokens in parallel which relieves the memory bandwidth b[ottleneck]." — the founder & CTO of Vulnetic

The skeptics: an old rumor and a metrics quibble

A senior data scientist who'd tracked Google's diffusion demos noted the lingering question — the technique was demoed a year ago and went quiet, reportedly because it was too expensive to run, "so that shouldn't be the case" given the new H100 comparison; the open question is "what the downside for this speed is here aside from being slightly weaker." Another reader flagged a benchmarking sleight: conflating throughput with latency, since a one-word reply still pays for "the entire 256 tokens as a chunk." Simon Willison, true to form, got it drawing a pelican via NVIDIA's free endpoint. The consensus the thread arrived at: not a frontier-quality model, but a credible bet that for local and edge inference, parallel token generation is the lever that matters.