Jane Street says formal methods are programming's future — and a team that banned JSON proved it with Lean 4

Why a trading firm thinks proofs are about to matter to everyone

Jane Street's essay (196 points, 73 comments) makes a specific bet about where programming is heading. The firm runs huge OCaml codebases where the code implements deterministic algorithms, so the mapping from spec to behavior is close to 1:1 — exactly the terrain where formal methods pay off. Their argument generalizes it: as generative AI produces more code than anyone can review, the scarce human skill shifts from writing code to specifying and verifying it. The post surveys their in-house work — the OxCaml prover, plus the wider industrial lineage of seL4, Dafny, TLA+ and Iris — as evidence that proof is moving from academia into shipping software.

The thread that followed was unusually credentialed: a former proof-of-correctness researcher, Lean and Coq users, and engineers running these tools against LLM output today.

"seL4 is verified and still had gnarly bugs"

The first asterisk came from a reverse engineer, puncturing the silver-bullet read:

"seL4 despite being formally verified contained gnarly bugs… Thus, it's not a silver bullet as some people think. Yes, it improves the quality, but only one of the aspects of hardening software, like ASAN, SAST, fuzzing…" — xvilka, a reverse engineer

Another reader sharpened where the risk actually moves: the seL4 spec is written in Isabelle and is itself complex, so "it's critical to be certain about the spec." Verification doesn't remove doubt — it relocates it from the code to the specification.

The objection a practitioner has carried for years

The deepest crack came from someone who keeps circling the field and bouncing off:

"Whenever I read about formal specs it always seems to me like 'write the same tests just in a different way', or worse, 'write the same implementation but in a different way'… I keep feeling like I'm missing something." — brap, who has looked into this on and off for years

A reader with 2.9k karma reinforced the boundary: deductive reasoning only helps where the postulates fit the domain. "Jane Street maintains large code bodies where the mapping is 1:1… But millions of us don't: most UIs, most exploratory work." That is the unspoken qualifier on Jane Street's whole thesis.

The engineer who banned JSON — and the veteran who warned about the trap

Then came the believer, describing a workplace that took it to the limit:

"We've banned everything from JSON to Python, rewritten nix to have a compiler, and almost everything we write is not only property tested and multiply fuzzed… but we have proofs in lean4 that at a minimum drive property tests via .olean linkage…" — reinitctxoffset

Against that maximalism, the most weight came from John Nagle, who did proof-of-correctness work decades ago. His warning is from inside the discipline, not outside it:

"What tended to go wrong with formal methods was that the people doing them were too into the formalism."

He noted his old systems had more proof automation than many later ones — the Oppen-Nelson simplifier and the Boyer-Moore prover — and that the hard human job was suggesting the right lemmas. That job is exactly what LLMs are now starting to do.

The AI plot twist: ChatGPT closing proofs a domain expert couldn't

The live reversal is that the tooling Jane Street describes is being changed mid-essay by the same AI wave that motivates it. A reader with real domain experience reported frontier models completing manual proofs faster than he could:

"ChatGPT can often prove something in minutes and 10 - 100 iterations that would take me, a human who has limited but non-zero proof assistant experience but significant domain experience in the lemmas being proven, much much longer." — addaon

A mathematician chose Lean 4 over OCaml for current research precisely because LLMs can now help write Lean, calling it a language that "makes OCaml read like Old English" — and gently suggesting Jane Street should migrate there too. Others reported pushing strong types in Scala 3 to carry compile-time proofs and keep AI agents from "falling into lower-quality modes."

So the real shape of the debate isn't "formal methods, yes or no." It's that AI simultaneously creates the problem (unreviewable code) and dissolves the historic blocker (proofs were too expensive to write). If a model can supply the lemmas that used to be the bottleneck, the spec becomes the only thing left for a human to get right — and as the seL4 voices warned, the spec is now where every remaining bug will hide.