Noise has been a great read with some very practical take-aways. As a natural sequel to the popular book ‘Thinking Fast and Slow’ it covers the topic of errors in our decision making and judgements using a range of real-world examples.
In the first book they go into depth on the different types of biases that affect our decision making. In this book they cover the second large element of error in decision making – noise.
Say you’re a manager and you need to fill a vacancy. Out of 100 candidates you and a panel of interviewers need to decide who to hire, with the goal being to choose who will be the highest performer. Decisions like this have an element of error – you may not choose the right person. This error can be broken down into two parts:
- Bias: when we consistently make errors in our decisions
- Noise: a random element in our decision making which can be driven by a variety of factors
The two are distinct and sum to total error in our decisions. Examples of bias include putting too much emphasis on the first impression of candidates or having a positive bias towards individuals from the same socio-demographic group as ourselves. In both cases this can lead to poor hiring decisions.
The noise element will be things like the interviewers mood, level of fatigue and differences in response to their own biases (e.g. the first impression bias was stronger for one person over another and across interviewers).
You can think of noise as the factors that can lead to variations in a decision of the same case by the same person. For example, the exact same interview may be judged differently by the same person, depending on whether it was before or after lunch. This is due to their mood.
Noise is clear as day in the data. If there were three interviewers and 100 candidates, you would see it in the variation of scores across the interviewers. Anyone who has done recruitment will be familiar with this at the moderation stage – when we look over our individual scores and discuss the differences.
Sources of noise
A lot of work has been done around biases, but in many cases noise can be a larger element in the decision making.
Noise can be broken down into:
- Level noise – say an interviewer is judging more harshly than others, leading to lower scores. You would see this clearly in the data.
- Pattern noise – the otherwise disagreements in scores across the interviewers. This includes occasion noise, where the same interviewer disagrees with their own score depending on the occasion. The before and after lunch is a good example.
Different responses to biases – when they are not shared by everyone, when they are present to different degrees, and when their effects depend on different circumstances.
Three types of biases that operate in different ways include:
- Substitution biases – when we substitute easier questions to help us make a decision, this can lead to incorrectly weighting certain evidence.
- Conclusion biases – overlooking evidence or considering it in a distorted way.
- Excessive coherence – magnifying the effect of initial impressions and reducing the impact of contradictory information.
These three types of biases can all produce noise.
Matching coherence – we rely on assumptions to make decisions and match to a closest estimate based on information we have. An outside prediction is based on information we hold or believe. An inside prediction uses specific contextual information given to us. Combining these two elements of prediction can lead to better decisions.
For example, an independent test score (outside information) can be used with an interview (highly contextual) when making a final decision.
Groups – when decisions are made in a group context they can be a large source of noise. In making our decision we are influenced by things like who speaks first, last, with confidence, seating arrangements, who gestures at the right moment etc. all these factors affect the decision outcome.
Minimising noise
Principles to minimise noise:
- Think statistically and take an outside view of the problem – this avoids incorrect judgements being made based on the specific contextual information.
- Structure judgements into independent tasks – this prevents the problem of excessive coherence. This is important in interviews to ensure a relatively poor score on one section doesn’t influence your perception of the other sections.
- Resist premature intuitions
- Take independent judgements from multiple people and factor them in – this avoids group influences.
- Favour relative judgements, as they tend to be less noisy than absolute ones. This is because our ability to categorise objects on a scale is limited and our ability to make comparisons is much better.
Practical takeaways:
- Noise is often a larger component of error than bias and need to be considered even if things have been put in place to avoid biases.
- Be strategic with timing and context to avoid noise in a judgement going against you e.g. when you want a positive outcome in a conversations, meeting or interview. Think about the individual or group making the decision and make sure they are – in a good mood, not hungry, not tired etc. All things equal, the same decision may go your way if you avoid these things.
- Speak first in meetings and email chains if you want to have a greater influence on a decision. People are influenced by each other and group dynamics will then work in your favour.
- Similarly, if a decision isn’t going your way, then speak sooner rather than later to interrupt the bandwagon effect of early views. If you wait until last some people may have already decided.
- Always look your best and have effective first interactions – on any given day we have interactions with new people. You always want to have a good first impression as this is a massive source of bias for future interactions and decisions.
- Try hard to establish a good rapport at the start of an interview or meeting as it will work in your favour. Early perceptions are based mostly on a candidate’s extraversion and verbal skills, so develop these to land a great first impression.
Further reading:
- Noise: A Flaw in Human Judgment (2021)
- https://thedecisionlab.com/biases