"AI passed the CFA Level III exam": what the studies actually tested
Headlines say AI models now pass the hardest level of the Chartered Financial Analyst exam. The research behind them is real, and it is narrower than the claim: mock exams, an estimated pass mark, a few dozen questions and best-of-several prompting.
The CFA charter is one of finance's toughest credentials, and "AI passes the CFA" is a headline that writes itself. Two academic studies are behind most versions of the claim. Read together, they show real progress in AI financial reasoning, and why the word "passed" needs an asterisk.
2023: GPT-4 on Levels I and II
In October 2023, researchers from Queen's University and J.P. Morgan AI Research posted "Can GPT models be Financial Analysts?" (Callanan et al.). They ran ChatGPT and GPT-4 on five Level I mock exams and two Level II mock exams, using zero-shot prompts, chain-of-thought reasoning and few-shot examples.
| Model and setup | Level I | Level II |
|---|---|---|
| ChatGPT, zero-shot | 58.8% (±0.2) | 46.6% (±0.6) |
| ChatGPT, best setup | 63.0% (±0.2) | 47.6% (±0.4) |
| GPT-4, zero-shot | 73.2% (±0.2) | 57.4% (±1.5) |
| GPT-4, chain-of-thought | 74.0% (±0.2) | 61.4% (±0.9) |
Did it pass? The authors could not say for sure, because the CFA Institute does not publish its minimum passing score. Using estimates gathered from candidates online, they judged that GPT-4 would pass Level I in every setup, while for Level II the verdict was "pass" with chain-of-thought or few-shot prompting and "unclear" with zero-shot. ChatGPT failed Level II in every setup.
They also found where the models broke: Level II questions, built around case vignettes, require deciding which information matters, and calculation-heavy topics were harder than conceptual ones.
2025: frontier models on Level III
Level III is the hard one: case-based multiple choice plus essay answers on portfolio management and wealth planning. In a paper accepted at the FinLLM workshop at IJCAI 2025 (Shetty et al., revised September 2025), researchers from the Rochester Institute of Technology, NYU Stern and the startup GoodFin tested 23 models on mock Level III material.
The test set came from AnalystPrep, a commercial prep provider, because official post-2018 Level III questions are not public. It contained 60 multiple-choice questions in 10 vignettes and 43 essay questions in 11 vignettes worth 149 points. Essays were graded by GPT-4 and, separately, by a certified human grader. The overall score is the simple average of the multiple-choice and essay scores.
| Model | Multiple choice | Essay (AI-graded) | Essay (human-graded) | Overall |
|---|---|---|---|---|
| o4-mini | 75.00% | 83.22% | 79.9 | 79.1% |
| Gemini 2.5 Flash | 73.33% | 81.21% | 78.5 | 77.3% |
| Gemini 2.5 Pro | 76.67% | 75.17% | 83.2 | 75.9% |
| Claude Opus 4 | 73.33% | 76.51% | 81.9 | 74.9% |
| o3-mini | 75.00% | 73.83% | not graded | 74.4% |
The authors compare these with an estimated 63% passing threshold, taken from the prep site 300Hours, and conclude that frontier reasoning models pass the mock exams. Reasoning models averaged 73.71 on AI-graded essays against 61.87 for non-reasoning models, at roughly three times the processing time. Context: n: not applicable (estimated pass mark) · 2025 estimate · 300Hours, cited in arXiv:2507.02954 · margin: not disclosed
Five asterisks on "AI passed"
1. The pass mark is an estimate. Neither study had an official threshold. The 63% figure comes from a test-prep website, and the 2023 paper relied on candidate reports from Reddit. Context: n: not applicable (estimated pass mark) · 2025 estimate · 300Hours, cited in arXiv:2507.02954 · margin: not disclosed
2. The test is small. Level III results rest on 103 questions from one commercial provider. A real exam sitting is a different, unseen paper.
3. Best-of-several prompting. The 2025 table reports each model's best score across the prompting strategies tried, sometimes mixing a different strategy for each half of the exam. A real candidate gets one attempt.
4. Graders disagree. The human grader awarded an average of 5.6 points more than the AI grader on essays, and the two agreed exactly on about 70% of questions on average across models (per-model counts in the paper; no interval disclosed). The authors say the pass conclusion holds either way. Context: n=989 essay responses (43 questions x 23 models) · mock Level III · arXiv:2507.02954 · margin: not disclosed
5. Possible conflicts and contamination. One co-authoring organization, GoodFin, is a company. The authors argue the paywalled prep material makes training contamination less likely, which reduces the risk without ruling it out.
What it does show
The trend is clear and meaningful. In 2023 the best model was borderline on Level II; by mid-2025, several models scored comfortably above an estimated Level III bar on mock material, and the remaining gap was concentrated in essays that demand judgment. That matches what analyst benchmarks show: models are strong at structured knowledge and weaker at multi-step work, as in the financial modeling scores on the Finance Agent v2 leaderboard.
What an exam cannot show is whether a model gives sound advice to a specific person in a live market. The CFA tests knowledge and reasoning on prepared cases. It is not a track record.
What it shows / What it doesn't
It shows
- That GPT-4 performed at an estimated passing level on mock Level I exams in 2023, and was borderline on Level II.
- That by 2025 frontier reasoning models scored above an estimated pass mark on a mock Level III exam, with the gap between models mostly in essays.
- That reasoning models beat non-reasoning models on essay answers, at a higher time cost.
It doesn't show
- That any AI has passed an official CFA exam. All tests used mock material and estimated thresholds.
- Robust scores. The Level III test had 103 questions, no intervals, and best-of-strategy results.
- Whether these models give good investment advice. An exam is not a track record.
- Freedom from contamination. Paywalled questions lower the risk but do not eliminate it.
Why it matters
"Passed the CFA" is increasingly used to market AI financial assistants. The research supports a narrower statement: frontier models now handle exam-style financial reasoning well on mock material, and still vary most on open-ended judgment. When you see the claim, ask which exam, which pass mark, how many questions, and how many tries.
Primary sources
- VerifiedCan GPT models be Financial Analysts? An Evaluation of ChatGPT and GPT-4 on mock CFA Exams (arXiv:2310.08678) (arXiv / Queen's University, J.P. Morgan AI Research)Primary source; accuracy table, pass verdicts (Table 7) and passing-score discussion checked October 14, 2026.
- VerifiedAdvanced Financial Reasoning at Scale: A Comprehensive Evaluation of Large Language Models on CFA Level III (arXiv:2507.02954, v2) (arXiv / RIT, NYU Stern, GoodFin)Primary source; accepted at FinLLM@IJCAI 2025 per the paper. Dataset, Table 1, Table 2 and grader comparison checked October 14, 2026.
In our directories
How we used AI: AI tools helped research and draft this article. Every figure was then checked against the primary sources listed above before publication.
Not financial advice. For information and education only. AI model outputs and research results can be wrong and do not predict future returns. Nothing here is a recommendation to buy or sell any asset. See our disclaimer and methodology.