From Kahneman, Sibony, and Sunstein's book Noise
We get things wrong. But understanding the anatomy of the error can be more important than the judgment itself.
Errors can be thought of in two ways: bias and noise. Bias is when errors are in the same direction. Noise is variability in judgments that should be identical.
Noise can be good. Disagreements and contrarian thinking are essential ingredients to innovation—the market tests competing strategies. But there are many decisions where noise is a problem. Those of us who think analytically often believe that random errors cancel each other out. However, in performance ratings, judgment calls, and measurement, noise is highly detrimental to companies.
We can sometimes correct bias by examining the decision-making process.
What to look for in reducing decision-making bias?
- Planning fallacy - Did people question the sources when they used data? How wide were confidence intervals for uncertain numbers?
- Loss aversion - Is the risk profile of the decision-making team aligned with the company?
- Present bias - Do the factors that led to the decision align with the short-term and long-term priorities of the company?
- Anchoring - Were uncertain numbers a significant factor in the decision?
- Nonregressive prediction - Did the decision-makers make insufficient allowance for regression towards the mean while predicting from an imperfectly reliable predictor?
- Premature closure -Was there accidental bias in the choice of considerations discussed early on?
- Excessive coherence - Were alternatives fully considered? Was uncomfortable data suppressed?
- Initial prejudgments - Do any of the decision-makers stand to profit more from one conclusion than another?
But even we when eliminate bias, there still exists systemic noise in the system that causes wrong decisions. Kahneman describes two types of noise: level noise and pattern noise. Level noise is variation across individuals. In a performance review, some reviewers are more generous than others. That's level noise. Judgment scales are ambiguous ("on a scale of 1 to 10"), and words may mean different things to different people ("exceeds expectations"). Pattern noise is the difference in the personal responses of people to the same things. It could be due to differences in principles or values that a person holds, consciously or unconsciously.
How do you reduce noise?
- Measure noise
What's measured gets managed. Kahneman and the authors did a study on the level of noise in a company. The executives estimated differences ranged from 5% to 10% in the organization. The results were shocking. They showed that the "noise index" ranged from 34% to 62%.
- Structure judgments into several independent tasks
Divide and conquer. Breaking decisions up into independent tasks reduces the tendency for people to distort or ignore information that doesn't fit the emerging story. Structured interviews are a great way to put this into practice.
- Resist premature intuitions
A decision made after careful consideration is always better than a snap judgment. Kahneman suggests that professionals shouldn't be given information that they don't need and could bias them, and calls this sequencing the information.
- Favor relative judgments and relative scales
Scales that use comparisons are less noisy than absolute scales.
- Obtain independent judgments from multiple teams, and then consider aggregating those judgments
Group discussions often create noise. Averaging different independent decisions will reduce noise but may not tackle bias.
These lists were taken from Kahneman, Sibony, and Sunstein's book Noise: A Flaw in Human Judgment.