Anthropic open-sourced an AI bug-hunting harness, and tptacek called it a shop jig

What the harness does, before anyone judges it

The repository is a reference implementation, not a product: an end-to-end pipeline that turns Claude into an autonomous vulnerability researcher. The stages are concrete — Build the target into a Docker image with AddressSanitizer; Recon to partition the codebase into input-parsing subsystems; Find, where N parallel agents craft malformed inputs until a binary crashes 3-of-3 times; Verify in fresh isolated containers; Dedupe against known bugs; Report with an exploitability writeup; and Patch, which drafts a fix and re-validates it against the original proof-of-concept and the test suite. It ships interactive skills (/threat-model, /vuln-scan, /patch) and refuses to run its autonomous loop outside a gVisor sandbox with egress locked to the Claude API.

One housekeeping note the thread had to settle: the repo lives under the GitHub org anthropics, which confused a reader into thinking it was an impostor "Anthropics." It isn't — that's the real Anthropic's org.

The security veterans shrug, deliberately

The top comment is the story. Thomas Ptacek — a well-known name in application security — declined to be impressed, and explained why in a metaphor:

"The thing about things like this is that they're shop jigs. You can buy a crosscut sled if you really want to, but most woodworkers just make their own." — tptacek

His point: two years ago there was real cost to building your own fuzzing harness, but "you probably weren't doing AI vuln research two years ago." Today the jig is cheap to make, so a published one is a convenience, not a moat. The economic counter-read came from a commenter who sees the business model: for the strongest security use cases, AI vendors will prefer to sell the technique as a managed service rather than hand over raw output — and indeed Anthropic links its hosted "Claude Security" product right in the README.

Then there's the bill. Simon Willison pulled the rate guidance — roughly 10K input tokens/min and 2K output tokens/min per agent, scaling to about 10 agents per 100K ITPM — and guessed a real scan lands in the hundreds of dollars. A skeptic added the obvious caveat: it's only as useful as Claude is token-efficient at actually finding bugs.

The quiet shift under the shrug

The dismissive framing is itself the news. When the most respected people in the room treat an autonomous bug-finding pipeline as a make-it-yourself shop tool, the capability has stopped being exotic. The interesting question isn't whether Anthropic's harness is impressive — it's that "AI finds the memory-corruption bug, reproduces it 3-of-3, and writes the patch" is now table stakes you're expected to wire up over a weekend.