ClickHouse turned PgBouncer's one CPU core into sixteen for 4x throughput
PgBouncer, the connection pooler sitting in front of most production Postgres databases, has never been able to use more than one CPU core, a limitation baked into how it was written years ago. ClickHouse's Postgres team split it into a fleet of processes sharing one port, but the fix only worked once they solved a stranger problem: a client hits cancel, and the request lands on a process that has never heard of the query.
On a 16-vCPU box, one PgBouncer process topped out around 77,000 transactions per second at 256 clients while using under 8 percent of the machine. ClickHouse's Postgres team ran sixteen processes on the same port instead, using a kernel trick called so_reuseport, and throughput jumped to 336,000 — roughly 4x — with a fix for cancel requests that used to vanish into the wrong process. One reply in the thread was blunter: Yandex already ships a pooler built to do this natively, no patch required.
PgBouncer's one-core ceiling, and how ClickHouse broke it
PgBouncer sits in front of nearly every production Postgres database, holding open a small pool of real connections so an app's thousands of short-lived clients don't each open one directly. It has one structural limit, and it dates back to how the tool was written: a single PgBouncer process uses exactly one CPU core, no matter how many the machine has. Kaushik Iska, a Director of Engineering on ClickHouse's Postgres team, lays out the consequence on a 16-vCPU box: pile on enough client connections and the pooler itself becomes the bottleneck, capped at whatever one core can push, while fifteen other cores sit idle.
The fix ClickHouse shipped isn't a rewrite. It's so_reuseport, a Linux socket option that lets multiple processes bind the identical port; the kernel then spreads incoming connections across all of them. Clients keep talking to one address and have no idea a fleet is behind it. Each process gets its own slice of the two connection limits, max_client_conn and max_db_connections, so the fleet can't collectively oversubscribe Postgres the way sixteen independently-configured poolers would.
The catch: a cancel that lands on a process that never heard the query
Splitting one process into many breaks something Postgres clients rely on. A query cancellation arrives over a brand-new connection, and with so_reuseport the kernel is free to route that new connection to any process in the fleet, not necessarily the one holding the original query. The cancel lands on a process that has never heard of it, and silently does nothing.
Iska's fix is peering: the processes register with each other over a [peers] block keyed by peer_id, so a cancel that arrives at the wrong process gets forwarded to whichever one actually owns the session. It's a small addition to the config: so_reuseport = 1 plus a peer list pointing at each process's Unix socket, and it's the piece that makes running sixteen independent processes behave like one pooler from the client's point of view.
77,000 versus 336,000 transactions a second
To measure the gain, ClickHouse ran identical pgbench workloads in transaction-pooled mode against a single PgBouncer process and against a sixteen-process fleet on the same c7i.4xlarge box, then swept the client count from 8 up to 256:
| Clients | Single process (TPS) | Fleet (TPS) | Single: box CPU | Fleet: box CPU |
|---|---|---|---|---|
| 8 | 8,910 | 6,450 | 0.8% | 2.9% |
| 32 | 54,203 | 64,244 | 5.2% | 12.3% |
| 64 | 86,570 | 219,439 | 8.3% | 31.9% |
| 128 | 83,463 | 320,547 | 8.1% | 45.9% |
| 256 | 76,893 | 336,469 | 7.7% | 48.9% |
The single process peaks at 64 clients and then gets slower as more connections pile on top of it: 76,893 TPS at 256 clients is below its own 86,570 peak at 64. The fleet keeps climbing the whole way, ending at roughly 4x the single process's best number. CloudWatch's own EC2 metric told the same story from the outside: 16% box CPU utilization for the single process versus 60% for the fleet, on the exact same workload. Below 16 clients, Iska notes, the single process is actually a hair faster, because there isn't enough concurrent load yet for sixteen processes to earn their coordination overhead.
The thread: "just use Odyssey"
The technical framing didn't go unchallenged. A commenter who describes himself as a "Postgres Hacker" pointed to a pooler that already does multi-core scaling natively:
"Just use https://github.com/yandex/odyssey :) It's a scalable PgBouncer."
He wasn't alone in reaching for an alternative rather than ClickHouse's patch. A teenage programming enthusiast from the UK said he'd been running pgdog instead, "and it has worked really well for my needs" — a second production pooler built to spread load across cores from the start, rather than bolted onto a single-threaded design after the fact.
bradfitz, who built LiveJournal, memcached and OpenID before more than a decade at Google and now works on Tailscale, put the underlying complaint more bluntly than either alternative-tool suggestion:
"I'm 46 now. I remember being shocked at Postgres's heavy connection model when I was 23. I gather things haven't improved since?"
The so_reuseport fleet is a workaround for that same 23-year-old architectural choice, not a change to it — Postgres still wants one OS process per connection, and every pooling scheme, ClickHouse's included, exists to hide that cost from the app.
Not everyone accepted that the extra machinery was warranted in the first place. A commenter with close to 4,000 karma questioned why PgBouncer needs peering at all:
"Why can't it just forward the cancellation to postgres, which then responds to the cancelled query with an error instead of a response, and then the bouncer for that connection handles that error?"
Nobody from ClickHouse answered that one in the thread. If your app already leans on a framework with connection pooling built in rather than routing every request through an external pooler, none of this fleet-and-peering machinery is a problem you have — it's a problem you never signed up for.
Get breakdowns like this every day
The top Hacker News stories, unpacked — by email, Telegram, or RSS.
Nvidia's $2 Billion Stake Covers Just 6% of CoreWeave's GPU Spending
CoreWeave plans to spend $31 billion to $35 billion on Nvidia GPUs this year, and is burning nearly twice as much cash as it earns to do it. Nvidia's own equity stake in the company: $2 billion, a rounding error next to the debt CoreWeave is taking on to keep building.