Confidently wrong: what the research says about AI hallucinations on financial numbers

Every AI model that handles money questions will sometimes produce a number that is simply wrong. The useful question is not whether, but how often, when, and whether you can tell. Here is what published research has measured.

1. Without data, models invent prices

In November 2023, Haoqiang Kang and Xiao-Yang Liu of Columbia University asked language models for historical stock prices with no data source attached, and checked the answers against Alpha Vantage. In the zero-shot setup, Llama2-7B and Llama2-7B-chat produced mean absolute errors of $6,357.6 and $6,380.5 per price queried. Few-shot examples brought answers closer but still left a gap.

The same study found the larger commercial models often declined rather than guessed: on the financial acronym and terminology tasks, GPT-3.5 Turbo abstained on 56.5% of questions and GPT-4 on 42.3% (task sizes of 192 or 1,000 items depending on the task; no interval disclosed). The authors' fix that worked best was giving the model a way to fetch data, either retrieval or a tool that writes the query for a price database. nContext: n=192 or 1,000 items depending on task · 2023 tests · Kang and Liu, arXiv:2311.15548 · margin: not disclosed

Lesson: a chatbot's memory is not a price feed. Any price or historical figure should come from a connected data source, not from recall.

2. With the documents, models still miscompute

The FinanceBench study (Patronus AI, November 2023) gave models the whole filing. On its 150 expert-graded questions, Claude 2 was wrong 21% of the time and GPT-4-Turbo 17% (n = 150 each; intervals not disclosed). The authors describe answers that were superficially coherent, with extensive calculations and reasoning, and still wrong, and call them harder to catch for that reason. We explain the full study in what FinanceBench actually measured. nContext: n=150 questions per configuration · late-2023 models · Islam et al., arXiv:2311.11944, Table 2 · margin: not disclosed

Lesson: the right document is necessary, not sufficient. Arithmetic across line items is where plausible errors live.

3. Confidence is not a safety signal

The newest evidence is the most uncomfortable. In "Confidently Wrong" (arXiv, July 2026, revised October 13, 2026), Richard Zhe Wang of St. John Fisher University tested three open models, Qwen3-8B, Llama-3.1-8B and Gemma-2-9B, on FinQA and TAT-QA, two standard benchmarks of numerical questions built from real company reports.

He asked each question eight times. If all eight answers matched, the answer was labeled confident. A common shortcut in AI products is to trust exactly these answers.

Among the confident answers alone, 15% to 23% were wrong on FinQA, depending on the model. On TAT-QA, all three models were more accurate and the confident-wrong share fell to 4% to 7% (question counts for TAT-QA not tabulated in the paper). The models also differ in temperament: Qwen3-8B and Gemma-2-9B tend to over-commit, while Llama-3.1-8B tends to hedge. nContext: n=6,105 to 6,114 FinQA questions per model · paper v2, Oct 13, 2026 · Wang, arXiv:2607.11414, Table 1 · margin: not disclosed

Can the errors be caught? The usual cheap signals, such as token probabilities and asking the model whether its own answer is true, performed poorly on confident answers, with AUROC scores of 0.55 to 0.63 (0.5 is a coin flip, 1.0 is perfect). Probes that read the model's internal activations did better, at 0.68 to 0.77. The author adds two honest caveats: a random-probe placebo already scores about 0.63 under his test design, and the probes did not transfer between FinQA and TAT-QA, losing 0.07 to 0.16 AUROC. They are task-specific triage tools, not a general lie detector.

Lesson: "the model gave the same answer every time" does not mean it is right. Roughly one confident FinQA answer in five to seven was wrong in this study.

4. Agents hallucinate sources too

Hallucination also shows up in AI agents' research habits. The August 2026 FrontierFinance benchmark found that some frontier models navigate straight to web addresses from memory, such as sec.gov or FRED pages, instead of searching. Those remembered URLs had significantly higher error rates, from hallucinated addresses or pages that block crawlers, which wasted effort and filled the agent's context with errors.

Lesson: a citation in an AI answer is only useful if the link opens and says what the answer claims.

What reduces the risk, according to these studies

• Connect data, do not ask memory. Retrieval and query tools cut price errors in the Columbia study.
• Prefer abstention. A model that says it cannot find the figure is safer than one that fills the gap.
• Recompute anything derived. Margins, growth rates and per-share numbers are where full-document setups still failed.
• Open the source. Remembered links fail more often than searched ones.
• Do not treat consistency as accuracy. Eight matching answers can still be eight wrong answers.