WallStreetDebug>_ Wall Street, debugged.
Markets
Subscribe
AI research

Tell a chatbot you are upset and its portfolio gets safer, a new experiment finds

Same finances, same stated risk tolerance, different tone. A preregistered experiment on three leading chatbots says the portfolio itself changed, not just the explanation. We read the abstract and explain what it can and cannot support.

5,040Allocation requests across 3 models
-9.4 pts1Change in risky-allocation share after distress wording
3Models tested

A good human adviser is supposed to separate how a client feels from what the client has said they can bear. A new preregistered experiment asks whether chatbots do the same. The author, John Garcia of California Lutheran University, posted the paper on SSRN on October 15, 2026. We read the SSRN abstract page and the Crossref copy of the abstract. The 30-page PDF was blocked to our fetcher, so everything below comes from the abstract, not the full text.

The setup

Three models answered 5,040 structured allocation requests: Claude Opus 5.5, Gemini 3.8 Flash and GPT-6 Sol. In 4,320 of the requests, three hypothetical investors described a loss, an upcoming withdrawal, or both, either in neutral words or in distressed words. The financial facts were identical across versions. After a loss, every request restated an unchanged risk tolerance, so the investor had not said they wanted less risk.

The outcome was whether the model's allocation sat above a stock and bond volatility reference matched to the investor's stated tolerance. In plain terms: did the model hand out a riskier mix than the investor's own stated comfort level implied?

What changed

Distress cut the share of allocations above that reference from 12.2% to 2.8%, a drop of 9.4 percentage points.1 The result sits beyond the preregistered plus or minus 5 point band the author set for calling two conditions equivalent. The direction held under alternative thresholds and covariances, although the decrease shrank to between 2.4 and 7.2 points in those checks.

An exploratory breakdown, which is weaker evidence than the preregistered test, located the change in the loss-only state: the share fell from 36.5% to 8.5%.2

What the models said about it

All three models accepted the task without asking a clarifying question. The author also coded whether the explanations kept a risk warning. That rate ranged from 0.9% to 74.9% of explanations depending on the model.3 Whether warnings rose and fell with the actual risk of each allocation is, in the author's words, unresolved: two models barely varied their warnings, and the third model's interval is compatible with both tracking and not tracking.

One more detail is easy to miss. Two models warned more often when their own allocations were handed back to them unchanged than when they had chosen the allocation themselves. The abstract does not explain why, and we will not guess.

How to read it

The paper's own summary is that expressed feeling changes the portfolio itself, not just the explanation. We would add a careful reading. A safer allocation after a loss is not automatically a worse one, and the study does not say which allocation was right. The finding is about consistency: with the stated risk tolerance held fixed, the output moved with the tone. For a tool that is supposed to be applying a stated profile, that is a reproducibility problem. For a tool meant to be sensitive to the client, it may be a feature. The experiment cannot settle which.

Limits you should keep in view

The investors are hypothetical, so nothing here measures what real clients did. The volatility reference is the author's construction, and the abstract does not give its formula. We saw model names and versions only as the abstract lists them. The paper is a posted SSRN preprint, and we have not read the methods section, the prompts or the robustness tables.

A small version you can run

The design is easy to copy at small scale. Write one allocation request with fixed numbers and a fixed stated risk tolerance. Create two versions of the backstory, one neutral and one distressed, and run each several times on the model you use. Compare the mixes it proposes, and record the model name, date and exact prompt. That is a consistency check on a tool, not a recommendation about any portfolio.

What it shows / What it doesn't

It shows

  • That in this experiment, distressed wording moved three chatbots toward lower-risk allocations even when stated risk tolerance and financial facts were unchanged.1
  • That the models did not ask for clarification and varied widely in how often they kept risk warnings.
  • A design other researchers and practitioners can replicate with their own prompts.

It doesn't show

  • Whether the calmer or the distressed allocation was better for the investor. The study does not rule on that.
  • How real clients behave. The investors were hypothetical.
  • Anything from the full text. We read only the abstract.

Why it matters

If tone alone can reshape a chatbot's portfolio, then the same question can get different answers on different days for emotional reasons, not financial ones. Anyone relying on a chatbot for a profile-based answer should test it for exactly this kind of drift.

Data notes

Sample size · period · source · margin of error for each figure marked with a number.

  1. n=5,040 structured allocation requests, 3 models · posted Oct 15, 2026 · Garcia, SSRN 7615736 (preregistered experiment) · 97.5% CI -11.1 to -7.7 points
  2. n=loss-only condition within the same 5,040 requests · posted Oct 15, 2026 · Garcia, SSRN 7615736 (exploratory) · margin: not disclosed
  3. n=coded explanations from 3 models · posted Oct 15, 2026 · Garcia, SSRN 7615736 · margin: not disclosed

Primary sources

Your turn: what should we test next?

Questions about this test or study, or an AI tool you want us to put through the same checks? Reply to our post on X @WallStreetDebug or email [email protected]. The best ideas become next week's tests.

Keep reading

In our directories

How we used AI: We read the SSRN abstract page and the Crossref copy of the abstract on October 17, 2026. The PDF returned an access error, so the methods, prompts and tables were not read. All figures are the author's, as stated in the abstract.

Not financial advice. For information and education only. AI model outputs and research results can be wrong and do not predict future returns. Nothing here is a recommendation to buy or sell any asset. See our disclaimer and methodology.

LLM adviceExperimentsRiskBehavioral finance