Daniel Kahneman on Intuition and the Outside View

I had the privilege of attending another Santa Fe Institute “Risk Conference” at Morgan Stanley. There was a stellar lineup of accomplished speakers focusing on Old Wine in New Bottles: Big Data in Markets and Finance. The grand finale was “A Conversation with Daniel Kahneman” led by Michael Mauboussin. These two gentlemen are amongst the finest thinkers in finance and two of the most important influences in my effort to compound knowledge while remaining cognizant of my limitations. As Mauboussin is intimately familiar with the subject matter, he was the perfect person to elicit the deepest insights from Kahneman on the most important topics. Below are my notes, which are reproduced here in the form of a dialogue. When I started jotting these down in real-time, I had no visions of writing the conversation up in this form; however, I found myself writing an awful lot with the output resembling an actual transcript. I attempted to be as thorough as possible in keeping the language as consistent with the spirit of the spoken dialogue as possible, though this is hardly perfect. I apologize in advance for the lack of completeness and the tense shifts, but nonetheless I am delighted to share the following in hope that others will be able to learn as much from this conversation as I did.

Michael Mauboussin: When does intuition work or fail?

Daniel Kahneman: Intuition works less often than we think. There is no such thing as professional “expertise.” The Intuitions in chess masters develop with “big data” comes from experience. For people, the immediacy of feedback is especially important to learn the basis of expertise. When feedback comes closer in time to the decision, intuition tends to be a lot stronger. Gary Klein, author of The Sources of Power is hostile to Kahneman’s view. Together they studied the boundary between trustworthy and untrustworthy sources of intuition. Confidence of intuition is NOT a good guide of intuition. If you want to explore intuition, you have to ask “not how happy the individual is” but what domain they are working in. There are some domains where intuition works, and some domains where it does not.  You need to ask “did the individual have an opportunity to learn irregularities on the way to building intuition? In domains where a lot of people have equal degrees of high confidence, they often do not know the limits of their expertise. 

Mauboussin: People blend quantitative and qualitative intuition, but what about disciplined intuition? Is there a better structure to decision-making?

Kahneman: When you put human judgment against simple models, after reading Paul Meehl’s book which showed where the human has access to all of the data behind the model, the model still wins in making decisions. There are no confirmed counter-examples. Studied an interviewing system for combat units. Asked multiple interviewers to speak with each candidate with a focus on one topic only per subject. Previously the interviewers had experienced a looser system without restriction—one interviewer per subject, with a broad focus. Unfortunately the previous system had zero predictive value on subsequent performance. At first, when the interviewers were instructed on a “disciplined” focus/topical breakdown, they were furious. People like using their broad intuitions. The interviewers were given a rating scale of 1 to 5 in each area they were assigned to cover. Eventually we got the data on how performance turned out based on the revised interview process. It turned out that interviews done in this way had much better predictive value for subsequent performance.

The problem with intuitions is how they come too fast. They are subject to confirmation biases. If you look at just one thing independent of all else and reserve judgment until the very end, what ultimately comes to mind will be more valid than if you don’t have discipline. It’s important to stress the independence (focus on 1 topic) to resist and overcome associative coherence—aka the halo effect.

Mauboussin: Define regression to the mean and the problems with it (causality, feedback)? 

Kahneman: Regression is a familiar concept, but not well understood. We see articles like “Why do smart women marry men less smart than they are?” That is an effect without a cause. We can reformulate that question to say that “the distribution of intelligence in men and women is the same” but the sound/implication of the two statements is not equivalent. You have to rid yourself of causation in making such statements. There was a study of the incidence of kidney cancer which described it as mostly rural, Republican districts in the center and south of the USA. Why? Everyone has a theory. But, if you look at the areas where incidence is small, it’s the same answer—mostly rural, Republican districts in the center and south of the USA.  This is so because the rural counties have smaller samples (a lower “n”) so incidences of high and low are more pronounced.

Mauboussin: Talk about the inside vs outside view, and base rates…

Kahneman: Was involved in writing a textbook on decision-making without math for a high school curriculum. Asked the team: “when will we finish the book?” Everyone answered somewhere between 18 and 30 months. Asked another colleague how long it took to write other textbooks in similar situations. This colleague’s answer had been somewhere in the 18 to 30 month range. The answer: 1) not all textbooks ever finished, with somewhere around 40% of them having given up; and, 2) those that were completed all took more than 7 years.

There are two different ways to look at a problem: 1) make an estimate based on a plan and reasonable extrapolation of progress—the inside view. 2) Abstract to the category of the case and ask “what are its characteristics”—the outside view. Intuition prefers the inside view, while the outside view is non-causal and statistical. If you start your analysis from the outside view, with a known base rate, it gives you a fair anchor and  ballpark from which to work.

Mauboussin: People are optimistic. There was a story you told of a few product launch at a company. At what point do you balance optimism vs just giving up? Society wants risks and all the good things that come with them.

Kahneman: Entrepreneurs don’t take risks because they love risk. They do it because they don’t know the odds. They don’t fully appreciate the risks they are taking. Optimism is the engine of capitalism. When you look at big successes, it’s because someone tried something they shouldn’t have.

Everyone should wish their children be optimists. They are happier, persevere more. Though, I don’t want a financial advisor who is an optimist. 

Mauboussin: As we embrace big data, it suggests change. When baseball learned about Moneyball, scouts resisted. With loss aversion, how do you relate this with the degree to which people are willing to embrace big data?

Kahneman: Losses loom larger than gains. Disadvantages are more salient and heavily weighted. In the context of change, one thing is guaranteed: there will be losers and winners. We can know ahead of time that the losers will fight harder than the winners. Losers know what they will lose, winners are never sure exactly what they will gain. People who initiate change don’t appreciate the resistance they will encounter. When reform is done in the regulatory arena, the reforms often compensate the losers making change very expensive. The prescription is to take the outside view.

The endowment effect is strong. The selling price someone sets on a sandwich they already owns and possesses is higher than that same person would price one they do not own. Giving up is more painful than selling something. This is evident in the financial arena. Advisors are helpful, because when they do the selling on someone’s behalf they do not have the same possessive connection and there is no endowment effect. Loss aversion is emotional, so if you make a decision in an advisor role, you can do so without emotion.

Mauboussin: When we look at decision making in an organization, there is noise. What does “noise” mean and why does it matter?

Kahneman: We know why Meehl was right on formulas being better than judges. For example, there was a situation that for each judge, there was a model built to predict what the judge will rule based on their past decisions. You can then compare the judge’s actual decisions with the model. The model is better than the judge. This tells you why people are inferior to formulas. A formula always has the same output. People vary and vary over time. When x-ray readers are asked to view the same image two separate times, 20% of the time they conclude differently. That’s what noise is.

Many organizations have functionaries who decide, but in principle they are interchangeable (credit-rating agencies, etc.) We would want all people to be interchangeable. How many individuals would be random in their actions? 45-50% tend to be variable. That variability is costly. Noise is costly. Most organizations think their employees agree with each other, but they don’t. Experience doesn’t bring convergence, it brings increased confidence. Convergence and confidence are not the same. If a financial advisory asked their advisors to prioritize a list of clients, does each advisor list the same clients in order? Probably not. When there is no selection, noise is costly.

Mauboussin: Give us a synopsis of Philip Tetlock's Superforecasting.

Kahneman: His book Expert Political Judgment was very important. It looked at predictions 10 years after experts made them and concluded forecasters can’t do it. And, the more a forecaster thinks they can do it, they less they actually did. With that knowledge, Tetlock built an IARPA tournament with predictions that covered timespans 6 weeks to a few months out (see my notes from Tetlock’s talks at two past SFI conferences here). He ID’d the superforecasters (the top 2%), which included a wide range of experts and ability. Short-term prediction being possible isn’t revolutionary. What makes superforecasters? A mixture of the inside and outside view. Disciplined intuition. Independent judgment, collated. 

I am skeptical of applying these findings in the political area where political figures themselves take actions that can be deterministic and statements have to be crafted to multiple constituencies, but in the financial arena these findings are very interesting.