The Math That Powers Modern AI Was Invented Three Times, Separately

Beltrami, Jordan, and Sylvester all found the same matrix trick, and none of them knew about the others

Eugenio Beltrami published the singular value decomposition (SVD) in 1873, showing that any bilinear form could be reduced to a simple diagonal form by rotating its two coordinate systems separately. A year later, in France, Camille Jordan proved the identical result through his own route, with no evident knowledge of Beltrami's paper. In 1889, James Joseph Sylvester found it a third time, again independently. Three mathematicians, three countries, sixteen years: one idea, discovered as if it had never been discovered before.

That's the story G.W. Stewart tells in *On the Early History of the Singular Value Decomposition*, the 1993 paper that surfaced on Hacker News as a plain PDF link. Stewart, a numerical analyst, traces how the decomposition now used to compress images, rank recommendations, and reduce embeddings kept resurfacing in isolated pockets of 19th-century mathematics before anyone connected the dots. Léon Autonne extended it to complex matrices in 1902; Erhard Schmidt pushed it into infinite dimensions in 1907, applying the same idea to integral equations rather than finite matrices at all.

The version everyone actually computes with today came later still, in two separate steps. In 1936, Carl Eckart and Gale Young proved that truncating a matrix's SVD gives the best possible lower-rank approximation to it, the theorem underneath modern recommendation systems and lossy image compression alike. But proving a decomposition exists and computing it reliably on a real machine are different problems. The numerically stable algorithm now built into every linear algebra library didn't arrive until 1965, when Gene Golub and William Kahan published it. Between Beltrami's proof and a usable algorithm sat nearly a century.

The paper's dedication hides a leap-year joke about its own hero

Readers who skipped Stewart's dedication page missed the best line in the document. One reader read it closely enough to explain the joke to the rest of the thread:

"Gene Golub was a numerical analyst, and father of the practical singular value decomposition (his license plate read 'Prof SVD'), together with William Kahan (the father of IEEE 754 floating point numbers). And his birthday was February 29. (In other words, the article was on the occasion of Gene's 60th birthday)." — a longtime commenter who calls himself an "unrepentant ultracrepidarian"

Stewart's paper is dedicated "for Gene Golub on his 15th birthday," a pun that only works if you know Golub was born on a leap day, and that a career spent making the SVD computable had earned him a vanity license plate to prove it.

SVD didn't stop in 1965: it's inside the optimizers training today's models

The thread's sharper turn was about what the decomposition is doing right now. An engineer in the thread worked through the connection between the 1936 Eckart-Young theorem and a much newer piece of machine learning:

"If you set up a neural network w/o nonlinearities that goes U -> S -> V^T … and minimize it with gradient descent, you will end up with the same U, S, and V that an SVD gives you. In fact, this is basically exactly what a Variational Autoencoder is! … Way too few people realize this connection, and I wish it was taught in more ML courses." — an engineer in the thread

A different reply, from a commenter who publishes a math research newsletter, noted that modern training optimizers, Muon among them, work directly on the singular values of a model's weight matrices, not just its raw gradients.

And then there's what the SVD looks like on the ground, for someone who has used it for over a decade:

"The SVD seems to come up everywhere in my work in computer vision … Claude and Codex are now generating all my code for me now, and I see them spitting out SVD code frequently -- often for very special cases." — a computer scientist working in computer vision

None of that priority mattered enough to survive into the tools built on it. The model that now writes the SVD call for a computer-vision researcher doesn't cite Beltrami, Jordan, or Sylvester, and nobody in the thread thought to ask it to.