Measuring risk tolerance: If a job’s worth doing, it’s worth doing well

With stories of unsuitable investments hitting the headlines again, the grounds for recommending one investment over another need closer scrutiny. That risk-tolerance tests are an essential piece of this puzzle is already acknowledged. That these tests are often unfit for purpose is, sadly, not.

Measuring risk tolerance well is mostly about avoiding common mistakes. Be it measuring something other than risk tolerance, confusing the audience, or relying on guesswork or substandard psychometric science, most attempts to measure risk tolerance fail in at least one crucial way.

How not to measure risk tolerance

Although the need to access investors’ risk tolerance has been embedded in law for a long time, the means of doing so have failed to meet what should be non-negotiable scientific standards. Items that pass the audition for inclusion in an assessment should be simple, but the process by which they are chosen shouldn’t be. Too often, assessment-designers get this the wrong way around: combining overly complex statements with too little statistical rigour.

Many understand the necessity of psychometric tests: we know that investors can’t reliably isolate the factors that affect how they think about risk at any one time. However, few grasp the concept that not all psychometric tests are created equal. Poor design can drive a distorting wedge between what a test means to ask and what it actually does, resulting in unsuitable outcomes. For example, you may aim at "How much risk are you willing to trade-off for better returns in the long-term?", but end up hitting: "How do I feel about taking investment risk this morning?".

Getting risk tolerance right is a science. Getting it wrong is a serious - and potentially costly - error.

Top tips on how not to do it:

1. Confusing the measurement.
   • Don’t confound risk tolerance with other attitudes, behaviours, or personality dimensions - Short-term emotional responses to markets should be understood in order to be controlled, not be baked into a recommended solution.
   • Don’t confound risk tolerance with investment objectives - What an investor is aiming to achieve with their investments is independent of their willingness to take on the risk of worse outcomes in order to achieve those aims.
   • Don’t confuse hypothetical choices with optimal actions - Responses to choices between hypothetical gambles, lotteries, or portfolios result neither in stable outcomes nor absolute levels that can be meaningfully used; a forced choice from a limited menu may not reflect investors’ reality.
   • Don’t confuse past behaviours with optimal actions - Preferences ‘revealed’ through actions are not necessarily fundamental ‘preferences’ in a psychological sense.
   • Don’t trivialise risk tolerance into ‘games’ - Gamification is great for engagement; but gimmicky games trivialise risk tolerance, they do not test it. Form should follow function, not replace it; if you’re not measuring what you’re supposed to be measuring, the playfulness of your polish doesn’t matter.
2. Confusing the audience.
   • Don’t require numerical calculations or probabilistic reasoning - This is a psychology assessment, not a maths test. Any percentage sign is usually a bad sign. In truth, numbers on the whole should be avoided - numbers cause stress, and stress can cause psychometric assessments to be too much psycho and not enough metric.
   • Don’t use potentially ambiguous or context-dependent statements - What shows statistical reliability in one culture or time period may not do so in others. Questions with a social-status subtext can lead us to lie to ourselves.
   • Don’t require knowledge of investing - We want to test risk tolerance, not knowledge. Questionnaires – especially ones not thoroughly tested - can be cursed with their designers’ knowledge, a failure to understand what it’s like to not understand even ‘simple’ concepts like the difference between shares and bonds.
   • Don’t reference, or require, knowledge of current market conditions - We need to assess long-term willingness to trade-off risk against return, not current attitudes to risk and markets. We want to measure an internal tolerance for risks, not the current state of the external risks themselves.
3. Favouring guesswork over science.
   • Don’t require respondents to assess their own future feelings - For example, responses to “How would you feel after a 10% drop in portfolio value?” are dreadful predictors of one’s future emotional state. An investor’s psychology is shaped by a lifetime of experience; it should be measured on that experience, not on speculation about what might happen in future, especially in reaction to a new event.
   • Don’t elicit returns expectations - Asking an investor what return they expect to experience creates an unreasonable expectation where none existed before. Return expectations are better ‘owned’ by the adviser and brought up after risk tolerance has been assessed.
   • Use only statements that discriminate effectively between individuals - Risk tolerance is a relative measure - we need people to be well-distributed across a range. If the majority of responses fall within a narrow band, the question is about as helpful as an abundance of straight-A scorecards is to a university admissions tutor.

The ideal solution dodges these traps and the risk of unwittingly failing the investor in the process. If a job’s worth doing, it’s worth doing well.

Greg B Davies, PhD - Head of Behavioural Science, Oxford Risk

Greg is a specialist in applied behavioural finance, decision science, impact investing, and financial wellbeing.

He started the banking world’s first behavioural finance team at Barclays in 2006, which he led for a decade.

In 2017 he joined Oxford Risk to lead the development of behavioural decision support software to help people make the best possible financial decisions.

Greg holds a PhD in Behavioural Decision Theory from Cambridge; has held academic affiliations at UCL, Imperial College, and Oxford; and is author of Behavioural Investment Management.

Greg is also Chair of Sound and Music, the UK’s national charity for new music, and the creator of Open Outcry, a ‘reality opera’ premiered in London in 2012, creating live performance from a functioning trading floor.

Back to Embark news

The information, materials or opinions contained on this website are for general information purposes only and are not intended to constitute legal or other professional advice and should not be relied on treated as a substitute for specific advice of any kind.

We make no warranties, representatives or undertakings about any of the content of this website (including without limitation any representations as to the quality, accuracy, completeness or fitness of any particular purpose of such content, or in relation to any content of articles provided by third parties and displayed on this website or any website referred to or accessed by hyperlinks through this website.

Although we make reasonable efforts to update the information on this site, we make no representation warranties or guarantees whether express or implied that the content on our site is accurate complete or up to date.