DeepSeek open-sourced the inference engine that serves its own production traffic

What is in the box, and what was held back

DeepInfer started on vLLM and borrows from SGLang and TensorRT-LLM. It bundles the FlashMLA, DeepGEMM and DeepEP kernels, attention data parallelism, expert parallelism with load balancing, speculative decoding and batch-invariant, deterministic sampling. License: Apache 2.0.

It supports only DeepSeek-V4.1-Flash, on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. The README shows prefill benchmarks on 4 GPUs and decode on 64, but no numbers in text. Missing: a KV cache system, prefill/decode disaggregation and data-parallel load balancing, the pieces that make large-scale serving cheap.

"Strategic sabotage" versus "they won't buy it"

The top comment treated the release as a threat to American labs:

"There's no way from here for them towards trillion dollar valuations… Ban all open weight models! Cried the VCs." — wg0

Another commenter saw the same reaction to the first DeepSeek model and doubted investors share it: big US and European companies, they argue, won't run Chinese models because of security, regulatory and continuity worries.

A 400-person lab as the benchmark

A commenter called it "crazy that a ~400 person side project by a Chinese firm can put up such a fight", noting that most rivals are small labs or side projects of tech companies. Someone else pointed to a prediction market on Anthropic's valuation that had not moved on the news.

For anyone self-hosting, the practical question is the missing parts: without KV-cache management and disaggregation, this is the engine, not the whole serving stack.