AI trading strategies lose their edge in 20 years of backtests

Many papers about AI trading agents report impressive returns. A study presented at KDD 2026, one of the main data science conferences, asks a blunt question in its title: can LLM-based investing strategies outperform the market in the long run? Its answer, on the evidence it gathered, is mostly no.

What they retested

The authors, Weixian Waylon Li and Tiejun Ma (University of Edinburgh), Hyeonjun Kim (Sungkyunkwan University) and Mihai Cucuringu (UCLA and University of Oxford), built a backtesting framework called FINSABER. It pits two published open-source LLM agents, FinMem and FinAgent, run with GPT-4o and GPT-4o-mini, against rule-based strategies, forecasting models such as ARIMA and XGBoost, and reinforcement-learning traders.

Their starting point is how earlier work was tested. By the authors' count, FinMem was evaluated over 6 months on 5 stocks, FinAgent over 6 months on 6, and TradingAgents over 3 months on 3. Short windows on famous winners invite two well-known traps: survivorship bias (forgetting the companies that failed) and data snooping (picking the test that happens to look good).

The study also sets costs that earlier papers often skipped: a 3% risk-free rate and a retail commission of $0.0049 per share, with a $0.99 minimum per order. nContext: n: not applicable, study assumption · 2004 to 2024 backtests · FINSABER, KDD 2026, arXiv:2505.07078 v6 · margin: not applicable

Step 1: the original window

First, the team re-ran the agents on the window where they had looked strong, October 6, 2022 to April 10, 2023. Even there, the authors found that FinMem only consistently outperformed on Tesla, while traditional benchmarks stayed competitive or better on the other stocks.

Step 2: stretch it to 20 years

Next, they kept the same four stocks (TSLA, NFLX, AMZN, MSFT) and extended the test to 2004 through 2024. The paper's verdict: a longer horizon "significantly diminishes the perceived superiority of LLM investors." Tesla was the only case where the LLM agents clearly led on annual return. On Netflix, Amazon and Microsoft, buy and hold or other strategies matched or beat them.

Step 3: remove the cherry-picking

Finally, the authors stopped hand-picking stocks. They drew from historical S&P 500 member lists, including companies later delisted, and let four selection methods choose the stocks in each period: five at random, a momentum screen, a low-volatility screen, and the stock-picking agent from another LLM framework, FinCon. That produced 91, 84, 63 and 80 distinct stocks respectively.

• Random five: buy and hold, an ATR band rule and ARIMA beat FinMem and FinAgent on risk-adjusted metrics.
• Low-volatility screen: buy and hold posted the best Sharpe ratio (0.703), the best Sortino ratio (1.291) and the best annual return (7.898%). FinAgent's Sharpe ratio was 0.241. nContext: n=63 stocks · 2004 to 2024 · FINSABER, KDD 2026, arXiv:2505.07078 v6, Table 4 · confidence interval: not disclosed (paired t-tests in Table 5)
• Momentum screen: FinAgent posted the highest annual return, 13.950%, but with a Sharpe ratio of only 0.104 and larger drawdowns. nContext: n=84 stocks · 2004 to 2024 · FINSABER, KDD 2026, arXiv:2505.07078 v6, Table 4 · confidence interval: not disclosed

The authors' summary is that previously reported LLM advantages "deteriorate significantly under broader cross-section and over a longer-term evaluation."

Too shy in bull markets, too bold in bear markets

The regime analysis explains part of the gap. According to the paper, the LLM strategies were "overly conservative in bull markets, underperforming passive benchmarks, and overly aggressive in bear markets, incurring heavy losses." In other words, they missed the rallies and took the hits.

Bigger models are not the fix

The study also finds that larger models "do not reliably outperform smaller ones, and both are consistently bettered by simpler models like ARIMA on risk-adjusted metrics." The authors add a point that cuts in the agents' favor: models like GPT-4o may have seen parts of the historical data during training, which should help them, and they still did not beat traditional strategies under fair testing.

The authors' recommendation for builders is to focus on detecting market regimes and controlling risk rather than stacking more complex agent frameworks. Their data and code are public.