Five AI assistants quoted 3,167 statistics, and only 40% led back to a study that agrees
Trakkr asked ChatGPT, Perplexity, Gemini, Claude and Google AI Overviews 200 number questions, then followed every citation trail by hand.
Four statistics, including that 44% of salespeople give up after one follow-up, were quoted by all five assistants with no study behind them anywhere. Company blogs made up 54% of first citations, and only 37% of those were sourced.
On the thread, a commenter showed the same failure live: a 70% PHP figure repeated by every article on one exploit.
Where 1,285 sampled statistics ended up
Trakkr's research asked each assistant 200 questions that need a number (160 everyday, 40 famous), via API with web search on, on 25 October 2026. That produced 1,000 answers and 3,167 statistics; 1,285 of them were traced up to five pages deep and checked by AI and then humans.
- 40% reached a study that says the same thing.
- 8% reached the study but the number or context had changed.
- 40% stopped at a locked, gone or silent source, and 11% led nowhere.
- ChatGPT sourced 57% (1.3 statistics per answer); Claude 38% (6.8 per answer).
On 28 of 71 questions where all five gave a percentage, the assistants gave four or five different figures. Limits: one day, API answers rather than consumer apps, paywalled sources counted as unverifiable.
"Thinly veiled ad", and a live example
The first reply attacked the messenger:
"Just an thinly veiled ad for something called trakkr." — a commenter who dismissed the post
Another commenter agreed the article was weak but said the issue is real, with an example from the same week:
"One of the original php2shell announcement posts (which was clearly AI written) claimed that 70% of websites use PHP. This number has now been repeated by every single article about php2shell." — a commenter tracing the claim
The creator of SQLAlchemy put the skill in a different place: being "correctly critical of LLM output" applies to human-written things too (zzzeek, creator of SQLAlchemy).
Get breakdowns like this every day
The top Hacker News stories, unpacked — by email, Telegram, or RSS.
Martin Fowler wants nobody to let an LLM write the first draft
Fowler admits he may be unable to spot 'LLM Voice' reliably, yet his site now requires disclosure of any AI help with writing.