gwern

A contest asked writers to generate a short story entirely by AI, no human edits, and to submit the full prompt harness. The winner, 'The June,' drew groans in the thread over its metaphors.

Then judge gwern spotted the twist: a large share of entries secretly read as allegories about AI powerlessness. Critics counter that the contest really measured prompt harnesses, not writing.

Dan Luu describes a workflow where a support ticket becomes a PR automatically, and he'll ship without human review if the fuzzing coverage is deep enough. He borrows it from Centaur, a chip company that ran 1,000 test machines for 40 engineers and shipped under one serious bug a year, with no unit tests.

After 50 runs he also found the viral 'caveman mode' token trick gave no consistent edge.

The thread argues over whether property-based testing can really replace unit tests.

The scores landed mid-table: 59.8% functional, 19.0% security, with a record number of timeouts from extended thinking. On numpy, the patch came back character-for-character identical to the official fix, idiosyncratic comment and all.

Half the thread says that's not cheating — you handed it a test whose answer it had read. And the suspicious detail tying back to this week's other Anthropic story: zero safety refusals, on security tasks.