4,392 answers later: the AI political-bias map and the fight over the grader

What the trakkr map actually measures

The trakkr.ai report plots six chatbots on the familiar two-axis grid: economic left-to-right horizontally, libertarian-to-authoritarian vertically. The method is deliberately stripped down to imitate how most people meet these models. Each gets the same open question bank with web search off and no system prompt, the setup the author calls Condition A. Reasoning is switched off across the board: reasoning effort none for ChatGPT and Grok, a thinking budget of zero for Gemini, final-answer-only for Claude. Llama and DeepSeek aren't reasoning models.

The scale of the run is the part the headline hides: 61 questions, 12 runs per model per question, six models, 4,392 answers total, dated June 2026. Temperature is left at default rather than zero, on purpose, because run-to-run wobble is the signal, not noise. The axis score is a weighted average of each answer's signed stance, with purely factual items zeroed out so only values-laden answers move the needle.

Reference points come from political-science data, not vibes. Party positions are mapped from the Chapel Hill Expert Survey (CHES), population distributions from World Values Survey Wave 7, and democratic context for non-European parties from V-Dem. A low-cost classifier reads every raw answer for signed stance, confidence, refusal type, hedge count and loaded language; then a second judge from a different lab re-scores a sample. On 639 dual-scored answers the two graders correlate at 0.95, with a mean stance disagreement of 0.06, and the report publishes where they diverge most (DeepSeek 0.09, Gemini 0.00).

The leaderboard, copied exactly

Three numbers travel with each model: its lean (signed position on the economic axis), its stability (how often it holds the same position when a question is re-run), and its flexibility (how far it bends when told who it's talking to, i.e. sycophancy under a persona).

A second table, what they say vs what they do, asks each model which way it leans, then compares the answer to where it actually measured. Grok claims neutral but lands +0.36 further right; Claude lands +0.34 further left than it states; ChatGPT and Llama both say neutral and read left. Gemini is the only one whose self-report and measurement match at 0.00.

The model-to-party mapping is where the report gets specific, and contentious: ChatGPT closest to Germany's Greens, Claude and Llama to New Zealand Labour, Grok to Emmanuel Macron, and both Gemini and DeepSeek to Australian PM Anthony Albanese.

The thread's first move: you're grading the grader

The 241-comment discussion opened on the oldest problem in this genre, stated cleanly:

"The outcome is entirely dependent on how the responses to 'politically charged questions' are graded as left vs. right. You're mostly just examining a delta in biases between the model and the investigator." — an established HN account with 500+ karma

The site's creator, posting as the submitter, answered with the actual countermeasures rather than a shrug:

"Each model's position is scored against outside political-science data… The stance coding is done by a separate model with a published prompt + a second model from a different lab re-scores a sample and we publish where the two disagree. So not perfect but… one of the more defensible approaches." — the report's author

A terser reply reframed the whole exercise as a measurement problem, not a verdict:

"Your multimeter reads out voltage relative to the black terminal, it's your responsibility to find the ground plane." — a commenter who recast bias as a reference-point choice

The asterisk hiding in the hero chart

The single most-upvoted catch wasn't about politics at all. It was about pixels:

"The logos have faint grey lines pointing to their actual data points, conspicuously making Grok look very far right. You wouldn't know it from glancing at the chart, but they measured ChatGPT being further to the left than Grok is to the right." — a reader who inspected the chart's layout

Others confirmed it depended on screen size and z-index layering: the logos render at fixed size while their dots shift to avoid overlap, so on a phone Gemini's connector line goes invisible and Grok drifts. An engineer traced the exact rendering bug, and another verified on desktop that the logos sit where intended only when no dots are drawn. The numbers back the complaint: ChatGPT's −0.29 is a larger displacement than Grok's +0.21, yet the visual reads the other way.

Where the calibration breaks: the German parties

The sharpest substantive challenge came from someone reading the worldview tab against ground truth they knew cold:

"It lists the German FDP to be further right than the CDU (which is nonsense)… They also register 'Die Linke' as Libertarian-Left, which could not be further from the truth… I'd honestly put them into 'Auth-Left' territory." — a commenter who knows German party politics in detail

Similar objections piled up across borders: that the grid placed Macron right of Xi Jinping and Obama, that it called avowed socialists Sanders and Sánchez the most "libertarian" politicians on the board. Each one points at the same seam: the CHES-to-axis projection that the author leans on for legitimacy is exactly where readers with local knowledge see it misfire.

One caveat from the thread sticks even if every other complaint is resolved. The benchmark ran with reasoning off and no system prompt by design, which an account that checked the configuration noted is the default consumer experience but not the state of the art. The leaderboard's value as a forecast shrinks the moment a model is actually allowed to think.