A file format ships a WASM decoder inside every file, and HN argues archival sense

A format that brings its own decoder

F3 — the name expands to Future-proof File Format — is a columnar storage format pitched as a successor to Parquet and ORC, backed by an academic paper. Its distinguishing move: each file is self-describing and ships its own decoders. "Each self-describing F3 file includes both the data and meta-data, as well as WebAssembly (Wasm) binaries to decode the data. Embedding the decoders in each file requires minimal storage (kilobytes) and ensures compatibility on any platform in case native decoders are unavailable." A Wasm decoder takes encoded bytes and returns an iterator of Arrow buffers; the claim is that you can invent a new encoding tomorrow and old readers will still open the file.

The README itself became a running complaint. It leans on terms like rectifying "the layout shortcomings of the last-generation formats like Parquet" without saying which shortcomings, and a top commenter noted that two minutes in, he still couldn't say what the format was for or why he'd care. (For the record: columnar analytics data, like Parquet.) Even dang surfaced the prior 2025 discussion to give it context.

"Genius" — and the people who archive petabytes disagree

The self-describing idea won real admirers. An engineer who works on data connectors saw the appeal immediately:

"This bit is quite genius, rather than depend on a language-specific SDK/lib for working with the formats you can fallback to exported WASM methods if none exist." — Gavin Ray, an engineer working on data connectors

The counter came from people whose job is keeping data readable for decades. A full-stack engineer who runs Kafka and RabbitMQ at scale put the archival case bluntly:

"If I am archiving PBs of data for 10+ years, I don't want to rely on a WASM interpreter being available and performant in the future just to read a file. I want a dead-simple, heavily documented byte specification like Parquet… putting the decoding logic inside a WASM binary introduces an active execution layer into what should be cold storage." — an engineer who runs Kafka and RabbitMQ at scale

A skeptic walked through why the interoperability promise may be thinner than it sounds:

"An embedded VM that decompresses video to raw pixels doesn't magically let you play that video in a text editor… Each new format still needs to be handled in a format-specific way." — a commenter poking at the interoperability claim

The compatibility moat F3 has to cross

Someone who actually read the paper landed the most concrete critique, that the format optimizes the wrong axis and still loses the only fight that matters:

"The most widely used parquet version is the oldest version from 2013… so parquet itself couldn't even supplant parquet. If you want to improve on it, you need to bring some serious results… Its main result seems to be improved random access which is not the point of columnar storage, as columnar storage was invented to exchange random access for fast analytics." — a commenter who read the F3 paper

One high-karma voice tried to rescue the design by reframing its target. F3 makes a poor operational format, he conceded, but as an archival one the self-unpacking property is exactly the point:

"F3 seems to be a reasonable archival data format… OTOH F3 would be easy to decode into almost any of today's accepted formats, and likely to any of tomorrow's. That's where being self-describing and self-unpacking would be important." — nine_k

The whole thread, fittingly, kept circling back to one webcomic that nobody had to link twice: a new standard meant to unify the old ones just becomes one more standard. The unresolved question isn't whether embedding a decoder is clever — everyone granted that it is. It's whether WebAssembly will still be the safe bet in 2051, or whether a file written to outlive its readers should bet on nothing more exotic than documented bytes.