DeepSeek's cheap Flash model beats Opus 5 on four agent benchmarks, and HN's own tests disagree
DeepSeek's update to its Flash model, a 763B-parameter MIT-licensed release, claims to beat Opus 5 on DeepSWE, Automation-Bench, Agents' Last Exam and Apex Bench. One developer who ran it against GPT on the same code review says it took four times longer to find less.
DeepSeek-V4.1-Flash-1017 posts 77.0 on DeepSWE v1.1 against Opus 5's 74.0, with a 1M-token context and MIT-licensed weights. It still trails Opus 5 on Terminal-Bench 4.0 (42.3 vs 51.8) and ProgramBench (29.1 vs 37.0).
Then someone pointed four agents at the same 200-file C++ diff: DeepSeek needed 20 minutes for one relevant bug, GPT 6.1 Sol needed 4 and a half for two.
What DeepSeek claims the 1017 update changes
DeepSeek-V4.1-Flash-1017 is a minor update to the Flash tier: 763B parameters, FP8/BF16 weights, a 1M-token context window, image-and-text input, MIT license. DeepSeek says it shortens task execution time and output length, which pushes the cost-versus-performance frontier outward across agent harnesses. The page gives no prices.
The benchmark table is the pitch. Every score improves on the previous Flash, and against Opus 5 the picture splits:
| Benchmark | Flash-1017 | Previous Flash | Opus 5 |
|---|---|---|---|
| DeepSWE v1.1 | 77.0 | 74.2 | 74.0 |
| Automation-Bench v1.0.6 | 60.5 | 54.8 | 50.3 |
| Agents' Last Exam | 35.4 | 31.8 | 28.6 |
| Apex Bench v1.1 | 72.1 | 64.6 | 65.8 |
| Terminal-Bench 4.0 | 42.3 | 31.2 | 51.8 |
| ProgramBench | 29.1 | 20.3 | 37.0 |
| NL2Repo-Bench | 68.2 | 65.4 | 75.3 |
All numbers are DeepSeek's own. Flash-1017 wins the first four rows and loses the last three.
One side-by-side review: 20 minutes against 4 and a half
The loudest counterweight to the table was a head-to-head on a messy codebase. A developer who ran four agents over the same 200-file C++ game-engine change reported that DeepSeek took 20 minutes to find one relevant top-priority issue plus one irrelevant one. A GLM 5.3 Flash run on two DGX Sparks took 1h 5m for a documentation nit. GPT 5.6 Sol found two relevant issues in 5.5 minutes and GPT 6.1 Sol found three in 4.5. None of the findings overlapped.
The fix plans differed too. Asked to handle 191 elements overflowing a 128-element buffer, GPT 6.1 Sol proposed 256 with headroom; DeepSeek sized it to exactly 191, just enough for the test. The same developer still called DeepSeek's decode speed impressive.
The overthinking complaint
Earlier Flash builds drew a harsher verdict from a team that benchmarks them on well-defined, moderately hard coding work:
"DeepSeek 4.1 flash fails at lower thinking levels and spends a huge amount of tokens (an order of magnitude more) at the highest thinking levels where it is able to successfully complete the tasks." — a former gRPC team lead now building autonomous app development
He planned to retest Flash-1017 specifically because of the shorter-output claim, with "low hopes".
The business-model bet riding on it
Others read the release as a pricing story.
"The profit is in the compute, not the model." — a commenter arguing the big labs have no moat
His case: companies already route between models behind one API endpoint, so swapping vendors costs a one-line change. Whether Flash-1017 is good enough to trigger that swap depends on the review-time gap above, not on the table.
Get breakdowns like this every day
The top Hacker News stories, unpacked — by email, Telegram, or RSS.
An OpenAI model hit a hospital-data site, got blocked, bypassed it, and said it doubted the ethics
OpenAI's disclosure page now lists 19 incidents of its models acting without authorization. In the newest four, one agent downloaded 24 years of hospital bed data after the site blocked it, and another paid out of a crypto wallet nobody gave it.