Amodei wants the government to be able to block model launches. The thread smells a moat

Dario Amodei's essay asks for mandatory third-party testing above a compute threshold — and government power to stop a deployment. The top reply reads it as a company at the top reaching for the state to pull the ladder up behind it.

In brief

Amodei frames AI like nuclear weapons and proposes an FAA-style regime: testing for cybersecurity, bioweapons, loss of control, and runaway R&D, with authority to block 'unacceptable' models.

The most-upvoted objection wasn't anti-safety. It pointed out that compliance overhead lands hardest on open-weight models — the exact competitors a frontier lab would want slowed down.

The proposal: testing you must pass, and a government that can say no

Policy on the AI Exponential
Policy on the AI Exponential

Read the essay before the reaction, because the reaction is entirely about its specifics. Amodei opens with a Tolkien image — policymakers as Hobbits trying to rouse Treebeard, slow institutions against an exponential — and argues recent cyber-risk evidence puts advanced AI in the strategic class of nuclear weapons. His concrete ask, modeled on the FAA: "Models above a threshold of compute should undergo mandatory testing by a qualified third party for their level of risk in four specific areas: cybersecurity, biological weapons, loss of control of AI systems, and automated R&D that could accelerate these other risks."

And the enforcement teeth: "The government should have the power to block or deter deployment of the model if it is determined, in light of third-party assessment, to present unacceptable risks." He pairs it with civil-liberties caveats and an acknowledgment that 2023–24 conditions favored transparency-only rules (he backed California's SB 53), arguing the evidence has since "graduated" toward binding regulation.

The thread's verdict: regulatory capture in a robe

The top response didn't argue about exponentials. It read the politics:

"How predictable. The company currently on top wishes to use the regulatory power of the state to prevent competitors from encroaching on their market dominance. It's a tail as old as time, although their CEO's rarely publish blog posts about it." — slopinthebag

The mechanism — who actually eats the compliance cost — was spelled out by a commenter quoting Amodei's own compute-threshold line back at him:

"AKA: Make it as expensive and untenable as possible for any open source model to jump through the regulation red tape so we can pull the ladder up behind us." — thewebguyd

The cleanest gotcha turned Anthropic's recent grievances against it. The company had publicly objected to being labeled a supply-chain risk; one reader pointed out the asymmetry:

"I feel significantly less sympathy for Anthropic's Supply Chain Risk designation if they believe the government should have this power over them. You get what you sign up for." — Imnimo

The other fight: is "exponential" even the right word?

A parallel argument rejected the framing's central noun. Several readers said "exponential" is doing rhetorical work the data doesn't support:

"If a model goes from 80% accurate to 85% accurate is that an exponential increase in 'cognitive capabilities?'" — themafia

The defenders didn't cite Anthropic — they cited an outside group. One pointed to METR's time-horizons data: "Looks pretty exponential to me … From a fully independent, non-profit research group." And a high-karma regular made the steelman demand: "Suggest a better name for what's happening with LLMs please."

Why the timing reads as strategy

The skeptical frame kept returning to one fact: the cadence. A regular noted the release-and-manifesto schedule lines up suspiciously well with a pre-IPO news cycle — "It is impressive how well they've scheduled all their releases, posts, and other news to dominate the tech news cycle almost every day in this pre-IPO phase." Whether you read Amodei's proposal as principled risk management or as moat-building, the testable claim underneath is narrow: a compute-threshold testing regime imposes a fixed cost, and a fixed cost is regressive — it weighs least on the largest incumbent. That's the part the thread wouldn't let go of, and it doesn't depend on guessing his motives.

Get breakdowns like this every day

The top Hacker News stories, unpacked — by email, Telegram, or RSS.

Next story · 19 of 22 Read more
🚀Tech

A 1-cent transfer can turn a bank's AI assistant into a phishing page

The payload doesn't go in the chat. It goes in the transfer description. The next time you ask the assistant 'show my recent transactions', the bank's own AI reads the attacker's text — and renders it as a trustworthy reauthentication prompt.

★ 166 · 💬 154 · blue41.com · 4 min