Cursor's benchmark says its cheap model rivals the frontier. A rival benchmark says it scores 16
CursorBench 3.1 shows Cursor's own Composer 2.5 landing near the top coding models at one-tenth the price. The thread pulled up an independent benchmark where the same model scores 16 to GPT-5.5's 64.
On Cursor's numbers, Composer 2.5 hits 63.2% accuracy at $0.55 a task — the cheapest in the field, and framed as near-frontier. The presentation even flipped the cost axis so cheapest sits on the right, which several readers found conveniently confusing.
The rebuttal came with citations. On the DeepSWE benchmark — 'probably the hardest to game' — GPT-5.5 gets 64, Opus 4.8 gets 56, and Composer 2.5 gets 16. A developer who's used both models heavily called the parity claim 'absolutely farcical': faster, yes; anywhere near as good, no.
What Cursor's benchmark claims
CursorBench 3.1 is Cursor's own evaluation of AI coding agents on "ambiguous, multi-file tasks from real Cursor sessions" — codebase understanding, bugfinding, planning, review. It spans 36 model configurations and tracks three axes: accuracy, cost per task, and tokens/steps. The topline the presentation is built to sell is that Cursor's in-house Composer 2.5 delivers near-frontier accuracy at a fraction of the price:
| Model | Accuracy | Cost / task |
|---|---|---|
| Fable 5 Max | 72.9% | $18.02 |
| Composer 2.5 | 63.2% | $0.55 |
| Composer 2 | 52.2% | $0.56 |
Composer 2.5 is the cheapest entry in the field, at roughly one-tenth the cost of the premium models. One presentational choice drew fire on its own: the cost axis is flipped so the cheapest model sits on the right, which several readers found needlessly disorienting.
The thread brought a benchmark of its own
The most damaging reply didn't argue vibes — it brought third-party numbers:
"Cursor's benchmark finds that Cursor's model (Composer 2.5) is basically as good as Opus 4.8 max and GPT-5.5 xhigh… You look at the DeepSWE benchmark (which is probably the hardest to game at this point) and GPT-5.5 xhigh gets a 64, Opus 4.8 max gets 56, and Cursor 2.5 gets 16." — a reader citing Artificial Analysis and DeepSWE scores
| DeepSWE | Score |
|---|---|
| GPT-5.5 xhigh | 64 |
| Opus 4.8 max | 56 |
| Composer 2.5 | 16 |
Someone who uses both models daily was blunter:
"I've used both Composer 2.5 and GPT 5.5… extensively, and their claim that Composer 2.5 is anywhere close in performance to GPT 5.5 is absolutely farcical. It's faster, but it's nowhere near as good." — a developer who runs both in Cursor and Codex
A six-month Composer user described the same pattern from the inside — fast, but "very shallow" on hard tasks, as if it "just finds the cheapest way to deliver." The tidiest summary was four words: "Cursor's model excels at Cursor's benchmark; news at 11." None of this makes Composer 2.5 useless — cheap-and-fast is a real product. It makes the benchmark a marketing asset, and the useful reading is the inverse of the pitch: when a vendor's model tops the vendor's eval, the number you want is the one measured by someone with nothing to sell.
Get breakdowns like this every day
The top Hacker News stories, unpacked — by email, Telegram, or RSS.
Podman 6.0 rebuilt its networking stack. The thread was really about whether it can replace Compose
Podman's big release swaps its networking internals for Netavark, Pasta and nftables and adds experimental rootless port forwarding that keeps the real source IP. The teams that already switched came to talk about what made them leave Docker.