Same person, different advice: how one risk question can swing an AI adviser's stock allocation by 24 to 35 points

Robo-advisers and chatbots increasingly set a stock allocation from a short questionnaire. The key input is almost always one item: how much risk are you willing to take? Research cited in a paper posted to arXiv on October 16, 2026 says people do not answer that item the same way twice. The paper, by four independent researchers led by Soovadeep Bakshi, asks what that does to the advice an LLM gives.

The setup

The authors drew 800 households from the Federal Reserve's Survey of Consumer Finances panel, in which the same person answered the risk question in 2007 and again in 2009. Each profile held fixed the household's 2007 age, education, income, net worth and financial assets. The only thing that differed between the two versions was the risk answer: the 2007 one or the 2009 one.

Seven models from five providers then gave one-turn advice on the share of financial assets to hold in stocks: Claude Sonnet 5.5, Claude Haiku 5.5, GPT-6.1 Sol, GPT-6 Luna, Gemini 3.8 Flash, Grok 4.7 and Mistral Large 4. For the first 200 households, every prompt was also asked twice unchanged, as a control for plain sampling noise.

How much the advice moved

Across all 800 households, changing only the risk answer moved the recommended stock share by 10.5 to 16.3 points on average, depending on the model. Among households whose answer actually changed, the swing was 24 to 35 points. Asking the identical prompt twice moved the advice by only 0.6 to 4.2 points, so the movement came from the input and not from randomness.¹

The paper also reports that between 23% and 35% of households saw their advice move by 20 points or more, depending on the model.¹

The ranking was stable: Claude Sonnet 5.5 swung least and Mistral Large 4 most on average. The paper adds a rough outside reference. Real households whose answer differs by one level hold about 10.4 points more or less in equities, so the models reacted 1.9 to 2.7 times as strongly. The authors call that comparison weak, since households are inert and some of the gap may be good advice.

Is it the crisis, the wording or the sample?

The panel straddles the 2008 crisis, so some answer changes were real. To check, the team used a six-month retest from the Basel-Berlin Risk Study, with 171 participants and no crisis. There, the rating alone moved advice by 11.2 to 17.7 points, as much as in the two-year panel. Rewriting the prompt as a fiduciary instruction, a first-person client voice, or a stocks, bonds and cash split left the swing at 11.8 to 20.5 points on the first 100 households.³

Who feels it, and what helps

The variation was uneven and largest for clients under 40. The authors also tested three mitigations, and the results are a mixed bag.

• Telling the model the answer is noisy steadied some models, but shifted weight to the client's age. It also pulled cautious clients toward a generic middle: clients unwilling to take any risk were advised 22% stocks instead of 4%.³
• Three extra money questions reduced the swing for some models, with no statistically detectable change relative to how much advice differs across people.
• In a scripted test, one clarifying question removed nearly all the swing from an answer the client then corrected, from 33.5 points to 1.9 on average, while keeping the response to a change the client confirmed.

The first two bullets sit under one set of conditions.³

A benchmark from classical test theory

The authors also ask how much advice should move. If each model discounted the answer for its measured reliability, as classical test theory prescribes, swings would shrink by a factor of 2.4 to 3.3. That benchmark assumes a model's response when told the answer is very reliable is its true mapping from attitude to advice. It tests consistency with the model's own rule, not whether the rule is correct. At a higher assumed reliability of 0.8, six models stay above the benchmark by 1.3 to 1.5 times, while Claude Sonnet 5.5 is statistically compatible with it.

Try it yourself

You do not need the paper's code to see the effect. Describe one fictional person to a chatbot with identical age, income and savings. Run the prompt three times, changing only the stated risk tolerance from cautious to moderate to aggressive. Then rerun the cautious version twice. If the repeat changes little and the three versions swing widely, you have reproduced the paper's pattern on a small scale. It is a demonstration, not a measurement.