Best AI agent passes 10.8% of 50 professional tax jobs, and a Sonnet beats Opus

1,789 files, eight practice areas and a 95% bar

Surge AI's DAYJOB: Tax tests agents on 50 environments spanning individual, business, partnership, international and other work, with 1,789 workspace files and a median of 30 per assignment. An attempt passes only if it meets at least 95% of the task's criteria.

The failures are judgment, not arithmetic: one agent judged both closing dates in a condo sale but missed that eligibility could lapse around August 2027; another assumed a separating couple was unmarried in all five attempts despite records pointing to a possible earlier marriage.

The thread: users say the same, and one is building the rulebook

A US citizen abroad who has run their own return through Claude for years listed what worked and what didn't:

"…it is not good enough to dump all your paperwork into and expect a shiny return you don't need to check." — a US citizen abroad who has fed Claude their own draft returns for years

Another commenter is encoding the forms themselves:

"I'm up to an estimated 60% coverage of US federal tax situations so far." — a commenter building an open model of IRS forms and worksheets

The implication

The benchmark and the user report agree: models are good second readers and poor sole preparers. If an agent drafts your return, the missing piece it is most likely to drop is the date or fact you never mentioned.