What investor scenarios can and cannot tell you

Ask somebody whether they are a confident investor and they must interpret “confident,” remember their experience and compress a context-dependent judgement into one answer. Present a specific market fall, unfamiliar product or sustainability trade-off and ask what they would do, and the task changes.

Scenarios make context more concrete and responses more comparable. They can reveal a tension between a general statement and a specific choice. But unless money, consequences and real options are involved, the answer is not observed investment behaviour.

Why scenarios are useful

They standardise context

Every respondent receives the same information, decision and response options. That helps separate differences in answers from differences in how an interviewer happened to frame the question.

They reduce abstraction

“How comfortable are you with uncertainty?” requires a person to invent their own meaning. A concrete situation provides a shared reference point, even though it remains simplified.

They make comparisons possible

A scenario response can be compared with stated risk comfort, confidence or knowledge. A disagreement between signals does not prove deception or future behaviour; it identifies something worth explaining.

They can test decision conditions

Researchers can vary one feature, such as familiarity, downside, evidence quality or number of options, to examine how choices change. A production profiling instrument is not automatically an experiment, but experimental logic helps improve question design.

The central limitation: hypothetical bias

Real decisions involve consequences, timing, emotion, competing goals and information that a questionnaire cannot fully recreate. People may answer according to an ideal, underestimate how they would react under stress or treat imaginary money differently from their own.

A broad review by Haghani and colleagues examined hypothetical bias across choice-experiment research in applied economics, psychology and neuroimaging. It found mixed evidence about the size and direction of bias across domains and contexts. The issue does not make choice experiments useless; it limits how confidently their results can be translated into real-world behaviour.

Use language that matches the evidence

Small wording choices establish whether a method sounds trustworthy.

• Avoid: “We measure what investors actually do.”
  Prefer: “We examine how investors respond to standardised decision scenarios.”
• Avoid: “The answers reveal who somebody really is.”
  Prefer: “Comparing answers can reveal a possible gap between self-perception and scenario choices.”
• Avoid: “This profile predicts investment behaviour.”
  Prefer: “The profile provides a hypothesis that can be compared with later behaviour.”
• Avoid: “The scenario proves the cause.”
  Prefer: “The pattern helps prioritise explanations to investigate.”

What makes a scenario more useful

Good scenario design is disciplined rather than dramatic:

1. One decision at a time. If a question changes market conditions, product complexity and social influence simultaneously, the answer is hard to interpret.
2. Plausible context. The situation should be recognisable to the intended population without requiring specialist experience.
3. Balanced options. Avoid making the “correct” or socially desirable answer obvious through tone or detail.
4. Meaningful variation. Response options should represent distinct strategies rather than slightly different wording.
5. Comprehension checks. If understanding the setup is essential, verify it rather than assuming it.
6. Pilot and review. Examine whether respondents interpret the scenario as intended and whether unexpected skews appear across relevant groups.

How u impact uses scenarios

The current investor profile combines three sources: self-report, objective knowledge questions and four standardised hypothetical scenarios. Scenario items carry a higher weight in the competence and ambiguity scoring, while no single item determines the result.

Two comparisons look for possible calibration gaps:

• self-rated knowledge versus answers to objective knowledge questions;
• stated comfort with risk versus responses across decision scenarios.

The calculation is rules-based. Optional AI may explain the approved facts but cannot change the scores, profile or gap. After each survey project, and at regular intervals as new aggregate data becomes available, the team reviews item performance, profile distribution and edge cases, updating calibration where evidence supports a change.

How scenarios become more credible over time

Software tests can demonstrate that the classifier behaves consistently: a given input produces the expected output, edge cases are handled and the same answers return the same result. That is essential technical validation. It is not psychometric or predictive validation.

A stronger evidence programme asks additional questions:

• Content: do the scenarios adequately represent the intended concepts?
• Reliability: are results sufficiently stable when the underlying construct has not changed?
• Construct validity: do scores relate to other measures as theory predicts without simply duplicating them?
• Predictive validity: do scenario patterns help explain later observed decisions?
• Fairness: are differences across groups meaningful, or are wording and context producing avoidable bias?

Continuous calibration supports this programme, but calibration and validation are not synonyms. Calibration adjusts the model against accumulating evidence. Validation tests whether the interpretations and uses of its scores are warranted.