Daniel Kahneman on Intuition and the Outside View

I had the privilege of attending another Santa Fe Institute “Risk Conference” at Morgan Stanley. There was a stellar lineup of accomplished speakers focusing on Old Wine in New Bottles: Big Data in Markets and Finance. The grand finale was “A Conversation with Daniel Kahneman” led by Michael Mauboussin. These two gentlemen are amongst the finest thinkers in finance and two of the most important influences in my effort to compound knowledge while remaining cognizant of my limitations. As Mauboussin is intimately familiar with the subject matter, he was the perfect person to elicit the deepest insights from Kahneman on the most important topics. Below are my notes, which are reproduced here in the form of a dialogue. When I started jotting these down in real-time, I had no visions of writing the conversation up in this form; however, I found myself writing an awful lot with the output resembling an actual transcript. I attempted to be as thorough as possible in keeping the language as consistent with the spirit of the spoken dialogue as possible, though this is hardly perfect. I apologize in advance for the lack of completeness and the tense shifts, but nonetheless I am delighted to share the following in hope that others will be able to learn as much from this conversation as I did.

Michael Mauboussin: When does intuition work or fail?

Daniel Kahneman: Intuition works less often than we think. There is no such thing as professional “expertise.” The Intuitions in chess masters develop with “big data” comes from experience. For people, the immediacy of feedback is especially important to learn the basis of expertise. When feedback comes closer in time to the decision, intuition tends to be a lot stronger. Gary Klein, author of The Sources of Power is hostile to Kahneman’s view. Together they studied the boundary between trustworthy and untrustworthy sources of intuition. Confidence of intuition is NOT a good guide of intuition. If you want to explore intuition, you have to ask “not how happy the individual is” but what domain they are working in. There are some domains where intuition works, and some domains where it does not.  You need to ask “did the individual have an opportunity to learn irregularities on the way to building intuition? In domains where a lot of people have equal degrees of high confidence, they often do not know the limits of their expertise. 

Mauboussin: People blend quantitative and qualitative intuition, but what about disciplined intuition? Is there a better structure to decision-making?

Kahneman: When you put human judgment against simple models, after reading Paul Meehl’s book which showed where the human has access to all of the data behind the model, the model still wins in making decisions. There are no confirmed counter-examples. Studied an interviewing system for combat units. Asked multiple interviewers to speak with each candidate with a focus on one topic only per subject. Previously the interviewers had experienced a looser system without restriction—one interviewer per subject, with a broad focus. Unfortunately the previous system had zero predictive value on subsequent performance. At first, when the interviewers were instructed on a “disciplined” focus/topical breakdown, they were furious. People like using their broad intuitions. The interviewers were given a rating scale of 1 to 5 in each area they were assigned to cover. Eventually we got the data on how performance turned out based on the revised interview process. It turned out that interviews done in this way had much better predictive value for subsequent performance.

The problem with intuitions is how they come too fast. They are subject to confirmation biases. If you look at just one thing independent of all else and reserve judgment until the very end, what ultimately comes to mind will be more valid than if you don’t have discipline. It’s important to stress the independence (focus on 1 topic) to resist and overcome associative coherence—aka the halo effect.

Mauboussin: Define regression to the mean and the problems with it (causality, feedback)? 

Kahneman: Regression is a familiar concept, but not well understood. We see articles like “Why do smart women marry men less smart than they are?” That is an effect without a cause. We can reformulate that question to say that “the distribution of intelligence in men and women is the same” but the sound/implication of the two statements is not equivalent. You have to rid yourself of causation in making such statements. There was a study of the incidence of kidney cancer which described it as mostly rural, Republican districts in the center and south of the USA. Why? Everyone has a theory. But, if you look at the areas where incidence is small, it’s the same answer—mostly rural, Republican districts in the center and south of the USA.  This is so because the rural counties have smaller samples (a lower “n”) so incidences of high and low are more pronounced.

Mauboussin: Talk about the inside vs outside view, and base rates…

Kahneman: Was involved in writing a textbook on decision-making without math for a high school curriculum. Asked the team: “when will we finish the book?” Everyone answered somewhere between 18 and 30 months. Asked another colleague how long it took to write other textbooks in similar situations. This colleague’s answer had been somewhere in the 18 to 30 month range. The answer: 1) not all textbooks ever finished, with somewhere around 40% of them having given up; and, 2) those that were completed all took more than 7 years.

There are two different ways to look at a problem: 1) make an estimate based on a plan and reasonable extrapolation of progress—the inside view. 2) Abstract to the category of the case and ask “what are its characteristics”—the outside view. Intuition prefers the inside view, while the outside view is non-causal and statistical. If you start your analysis from the outside view, with a known base rate, it gives you a fair anchor and  ballpark from which to work.

Mauboussin: People are optimistic. There was a story you told of a few product launch at a company. At what point do you balance optimism vs just giving up? Society wants risks and all the good things that come with them.

Kahneman: Entrepreneurs don’t take risks because they love risk. They do it because they don’t know the odds. They don’t fully appreciate the risks they are taking. Optimism is the engine of capitalism. When you look at big successes, it’s because someone tried something they shouldn’t have.

Everyone should wish their children be optimists. They are happier, persevere more. Though, I don’t want a financial advisor who is an optimist. 

Mauboussin: As we embrace big data, it suggests change. When baseball learned about Moneyball, scouts resisted. With loss aversion, how do you relate this with the degree to which people are willing to embrace big data?

Kahneman: Losses loom larger than gains. Disadvantages are more salient and heavily weighted. In the context of change, one thing is guaranteed: there will be losers and winners. We can know ahead of time that the losers will fight harder than the winners. Losers know what they will lose, winners are never sure exactly what they will gain. People who initiate change don’t appreciate the resistance they will encounter. When reform is done in the regulatory arena, the reforms often compensate the losers making change very expensive. The prescription is to take the outside view.

The endowment effect is strong. The selling price someone sets on a sandwich they already owns and possesses is higher than that same person would price one they do not own. Giving up is more painful than selling something. This is evident in the financial arena. Advisors are helpful, because when they do the selling on someone’s behalf they do not have the same possessive connection and there is no endowment effect. Loss aversion is emotional, so if you make a decision in an advisor role, you can do so without emotion.

Mauboussin: When we look at decision making in an organization, there is noise. What does “noise” mean and why does it matter?

Kahneman: We know why Meehl was right on formulas being better than judges. For example, there was a situation that for each judge, there was a model built to predict what the judge will rule based on their past decisions. You can then compare the judge’s actual decisions with the model. The model is better than the judge. This tells you why people are inferior to formulas. A formula always has the same output. People vary and vary over time. When x-ray readers are asked to view the same image two separate times, 20% of the time they conclude differently. That’s what noise is.

Many organizations have functionaries who decide, but in principle they are interchangeable (credit-rating agencies, etc.) We would want all people to be interchangeable. How many individuals would be random in their actions? 45-50% tend to be variable. That variability is costly. Noise is costly. Most organizations think their employees agree with each other, but they don’t. Experience doesn’t bring convergence, it brings increased confidence. Convergence and confidence are not the same. If a financial advisory asked their advisors to prioritize a list of clients, does each advisor list the same clients in order? Probably not. When there is no selection, noise is costly.

Mauboussin: Give us a synopsis of Philip Tetlock's Superforecasting.

Kahneman: His book Expert Political Judgment was very important. It looked at predictions 10 years after experts made them and concluded forecasters can’t do it. And, the more a forecaster thinks they can do it, they less they actually did. With that knowledge, Tetlock built an IARPA tournament with predictions that covered timespans 6 weeks to a few months out (see my notes from Tetlock’s talks at two past SFI conferences here). He ID’d the superforecasters (the top 2%), which included a wide range of experts and ability. Short-term prediction being possible isn’t revolutionary. What makes superforecasters? A mixture of the inside and outside view. Disciplined intuition. Independent judgment, collated. 

I am skeptical of applying these findings in the political area where political figures themselves take actions that can be deterministic and statements have to be crafted to multiple constituencies, but in the financial arena these findings are very interesting.

A day with SFI learning "Optimality vs Fragility"

Recently I had the privilege of attending Santa Fe Institute's latest joint conference with Morgan Stanley. This time, the topic was "Optimality vs Fragility: Are Optimality and Efficiency the Enemies of Robustness and Resilience?" The topic was both intriguing and timely, and the speakers were interesting, informative and a little bit more controversial than in years past. This made for an outstanding day. The audience in the room included some big names in finance and science alike, setting the stage for fascinating Q&As and stimulating conversations during the breaks.

This year, rather than writing one big post covering all of the lectures, I will break each down into its own entry. Here are the subsequent posts in order (and their respective links). Let this serve as your guide in navigating through the day:

Cris Moore--Optimization from Mt. Fuji to the Rockies

Nassim Taleb--Defining and Mapping Fragility

John Doyle--Universal Laws and Architectures for Robust Efficiency in Nets, Grids, Bugs, Hearts and Minds

Rob Park--Logic and Intent: Shaping Today's Financial Markets

Juan Enriquez--Are Humans Optimal?

Dan Geer--Optimality and Fragility of the Internet

I like to think about are how the lectures relate to what I do in markets and where there is overlap and dissention between the speakers. Further, I like to analyze how some of these lectures fit (or don't) with my preexisting views. I would love to hear what others think. Here are a few of my observations to get you all started:

• Cris Moore's point that "best" is not necessarily optimal, and a confluence of models (what he calls data clusters) can yield better outcomes is extremely important in financial markets.
• Nassim Taleb's suggestion that stress tests should focus on accelerating pain, rather than spot analysis is a powerful one that all risk managers should think about.
• John Doyle's observation about the tradeoffs between robustness and efficiency is directly applicable to portfolio construction.
• Rob Park's explanation of how algorithms are designed to express human intent, and the areas in which that can go has me rethinking my understanding of the risks from HFT.
• Juan Enriquez opened everyone's eyes to how big the advances are in life science and the consequences this holds for the "secular stagnation" debate.
• Dan Geer's explanation for why we have a choice between two of "security, convenience and freedom" online is both an enlightening and frightening call to action.

Again I will caution that these are my notes from the sessions. There is no guarantee of accuracy or completeness. I specifically focused on points that were intriguing to me, and purposely left out areas where the subject matter and terminology were too far removed from my competency. 

Michael Mauboussin on the Santa Fe Institute and Complex Adaptive Systems

Michael Mauboussin of Credit Suisse is one of the best strategists on Wall Street and a thought leader who consistently introduces some of the most compelling topics to the financial community. It is therefore no surprise Mauboussin is now Chairman of the Board of Trustees at the Santa Fe Institute, an organization which specializes in the multi-disciplinary study of complex adaptive systems. I recently had the privilege of interviewing Mauboussin about his involvement with the Santa Fe Institute and his thoughts on complexity. Enjoy (and be sure to follow the links to some fascinating further readings):

Elliot: Now that you’re Chairman of the Board of Trustees at the Santa Fe Institute, what are your goals and visions for how to more broadly inject SFI’s lessons on complexity into the financial community’s understanding of markets?

Michael: In my role at SFI, the primary goal is to make sure that the Institute does, and can, do great science. The unifying theme is the study of complex adaptive systems. But the goal is to have a place where there’s support for important, transdisciplinary research. 

That said, I would love to continue to see this type of thinking work its way into our understanding of financial markets. That is happening to some degree. One example is Andrew Lo’s work on the Adaptive Market Hypothesis. Another example is Blake LeBaron’s work on markets using agent-based models. I think it’s a more complete way of viewing markets than a standard rational agent model or the assumption of the absence of arbitrage. The problem is that modeling complex adaptive systems is a lot messier than those other approaches.

Elliot: When we last met at an event introducing The Success Equation to SFI members in New York, I asked you what the right “success equation” is for a young investor. Your response was to “keep coming to these events.” How did you first learn about the Santa Fe Institute? And how did you come to embrace the SFI?

Michael: I first learned about SFI in 1995 at a Baltimore Orioles baseball game, where Bill Miller was my host and the proselytizer. He explained how this new research group dedicated to the study of complex systems was coming up with cool and useful insights about business and markets. Specifically, he was taken with Brian Arthur’s work on “increasing returns.” This work showed that under some conditions returns actually move sharply away from the mean. This is counter to classic microeconomic thinking that assumes returns are mean-reverting. 

In many ways I was primed for the message. I had been doing a lot of reading, especially in the area of science, and so this way of thinking made sense to me from the beginning.

Elliot: Did you have a bias towards one market philosophy before you adopted the complex adaptive system mental model?  

Michael: Although I had a solid liberal arts background before starting on Wall Street, I had very little background in business or finance. As a result, I had few preconceived notions of how things worked. It’s a challenge to come up with clear conclusions based on an observation of what happens in markets. On the one hand, you see clear evidence that some people do better than the indexes and that there are patterns of booms and crashes over the centuries. These suggest that markets are inefficient. On the other hand, there’s also clear evidence that it’s really hard to beat the market over time, and that the market is more prescient than the average investor. So for me, at least, there was an intellectual tug of war going on in my head. 

I have to admit to being struck by the beauty of the efficient markets hypothesis as described by the economists at the University of Chicago. At the forefront of this, of course, was Eugene Fama, who recently won the Nobel Prize in part for his work in this area. What’s alluring about this approach is that it comes with a lot of mental models. You can equate risk with volatility. You can build portfolios that are optimal relative to your preference for risk. And so forth. Because you can assume that prices are an unbiased estimate of value, you can do a lot with it. The market’s amazing ability to impound information into prices impresses me to this day.

So it was with this mental tug of war as a backdrop that I learned about the idea of complex adaptive systems. Suddenly, it all clicked into place. A simple description of a complex adaptive system has three parts. First, there are heterogeneous agents. These can be ants in an ant colony, neurons in your brain, or investors in a market. Second, these agents interact leading to a process called “emergence.” The product of emergence is a global system that has properties and characteristics that can’t be divined solely by looking at the underlying agents. Reductionism doesn’t work. 

What instantly drew me to this way of thinking is that it describes markets very well and it is very common in nature. The central beauty of this approach is that it provides some sense of when markets are likely to be efficient—in the classic sense—and when inefficiencies will creep in. Specifically, markets tend to be efficient when the agents operate in a truly heterogeneous fashion and the aggregation mechanism is working smoothly. Diversity is essential, both in nature and in markets, and the system has to be able to take advantage of that diversity. There are some neat examples in experimental economics to show how this works. It’s really wondrous. 

On the flip side, when you lose diversity the system can become very inefficient. And that’s also what we see in markets—diversity loss leads to booms and crashes. Now the loss in diversity can be sociological, in which we all start to believe the same thing, or it can be technical, such as the winding up or winding down of a leverage cycle. But here we have a framework that accommodates the fact that markets are pretty darned good with the fact that they periodically go haywire. And SFI was at the center of this kind of thinking.

Elliot: It’s interesting that your answer on what theory of markets you subscribe to is not in the “black or white” vein whereby one must be in one camp and one camp only. It seems like much of the divisiveness in today’s discourse (in many arenas) stems from people’s unwillingness to see these kinds of shades of grey, though as you suggest, that mentality is not for everyone.  Do you meet resistance from people when explaining your stance? Is there a way to get others to embrace “complexity” when people have an innate desire for linear, orderly explanations that are essentially either/or answers? 

Michael: Most of us are uncomfortable with ambiguity—we’d rather just have a point of view and stick to it. But in markets, the real answer clearly lies between the folks who believe that markets are perfectly efficient and those who believe it’s largely inefficient. By the way, if you think the market is mostly inefficient there is no reason to participate because even if you have a sense that you are buying a dollar at a discount there is no assurance that the market will ever recognize that value. So some degree of market efficiency is essential even for those who believe that markets have inefficiencies. 

My goal is less to get people to change their view and more to establish a better understanding of how things work. Once you learn about markets as a complex adaptive system and appreciate its implications, I find it difficult to go back to a more traditional point of view.  

Elliot: In More Than You Know, you said, “The best way to describe how I feel following a SFI symposium is intellectually intoxicated.” Are there steps you take following these events to transform the ideas you’ve learned and the relationships you’ve built into expanding the scope of your own knowledgebase? And how are you able to harness this intoxication into productive output? 

Michael: I wish I could be more systematic in this regard, but I think it’s fair to say that the ideas from SFI have permeated every aspect of my work. Perhaps a couple of examples will help make the point.

I’ve already mentioned conceptualizing markets as a complex adaptive system. This alone is a large step, because rather than simply moaning about the limitations of standard finance theory, you have a framework for thinking about what’s going on.

I’ve also already mentioned Brian Arthur’s work on increasing returns. Many businesses are being defined less by their specific market segment and more by the ecosystem they create. And it is often the case that in a battle of ecosystems, one will come out on top. So this set of steps provides a mental model to understand the process of increasing returns and, as important, how to identify them in real time.

Ideas from SFI have inspired my work in many other ways, from understanding power law distributions in social systems to network theory to collective decision making to the processes underlying innovation. I could go on. But suffice it to say that there is hardly an area of markets, business, or decision making where your thinking wouldn’t be improved by learning, and internalizing, the kinds of ideas coming out of the SFI.

Elliot: In More Than You Know, you also introduce Charlie Munger and SFI as “Two sources in particular [that] have inspired my thinking on diversity. The first is the mental-models approach to investing, tirelessly advocated by Berkshire Hathaway's Charlie Munger. The second is the Santa Fe Institute (SFI), a New Mexico-based research community dedicated to multidisciplinary collaboration in pursuit of themes in the natural and social sciences.” It seems only natural that adopting Charlie Munger’s perspective to mental models would lead one to the SFI. Can you talk about the synergies between these two worldviews in making you a better analyst? What role did your adoption of Munger’s framework play in your attraction to the SFI?

Michael: Charlie Munger is a very successful businessman. Probably the first thing to note about him is that he reads constantly. He’s a learning machine. There’s bound to be a good outcome if you dedicate yourself to reading good stuff over a long period of time. That alone should be inspiring.

So as I think about the synergies between the worldviews, a few thoughts come to mind. First, it’s essential to provide your mind with good raw material. That means exposing yourself to a lot of disciplines and learning the key tenets. It also means spending time with people who think differently than you do. 

Second, you have to be willing and able to make connections. What are the similarities between disease and idea propagation? What can an ant colony teach me about innovation? What do physical phenomena, such as earthquakes, tell us about social phenomena, such as stock market crashes? You need good raw material to make connections, but you also have to be careful to avoid superficial links.

Finally is the idea of thinking backwards. Munger is a big advocate for this. You observe that something is where it is: How did it get there? Why did it get there? There are some fascinating challenges in this regard right now. We know, for example, that the sizes of cities and companies follow power laws. Why? By what mechanism does this happen? No one really knows, and the prospect of solving those kinds of challenges is exciting.   

But I have to finish with the point that this approach to the world is not for everyone. The interest or capability to work in this fashion is far from universal. So I wouldn’t recommend this to everybody. Rather, I would encourage it if you have a proclivity to think this way.  

Elliot: You talk of a benefit of the mental models approach as having a diverse array of models that you can fit a given situation, rather than fitting a given situation to a one-size-fits-all model.  Can you shed some insight on a) how you built up your quiver of models; b) how you organize these models (either mentally or tangibly); and c) how you choose which model to use in a given situation?

Michael: Yes, I think the metaphor is that of a toolbox. If you have one tool only, you’ll try to apply it to all of the problems you see. And we all know people who are just like that.

The mental models approach seeks to assemble a box with many tools. The idea is to learn the big ideas from many disciplines. What are the main ideas from psychology? Sociology? Linguistics? Anthropology? Biology? And on and on. In many cases you don’t have to be a deep expert to get leverage from a big idea. One of my favorite examples is evolution. Spend some time really understanding evolution. It is a mental model that applies broadly and provides insights that other approaches simply can’t.

I’m not sure I’m much of an example, but I have strived to read widely. This in part has been inspired by the people and ideas I have encountered at SFI. Most of my organization comes through writing or teaching. For me, that is a way to consolidate my understanding. If I can’t effectively write or teach something, I don’t understand it. Now I’m sure I write about things I don’t understand as well, but I try my best to represent the science as accurately as possible.

As for choosing the right model, the key there is to look for a fit. One concept that intrigues me is that nature has taken on and solved lots of hard problems, and there’s a lot we can learn from observing how nature works. So you might learn how to run a committee more effectively if you understand the basic workings of a honeybee colony. Or you might have insight about the resources your company should allocate to experimentation by examining ant foraging strategies. 

The risk is that you take the wrong tool out of the toolbox. But I think that risk is a lot smaller than the risk of using the same tool over and over. I’ll also mention that the work of Phil Tetlock, a wonderful psychologist at the University of Pennsylvania, suggests that so-called “foxes,” people who know a little about a lot of topics, tend to be more effective forecasters than so-called “hedgehogs,” those with a single worldview. So not only is this an intellectually appealing way to go, there’s solid evidence that it’s useful in the real world. 

Elliot: When you cite how Brian Arthur’s work “showed that under some conditions returns actually move sharply away from the mean. This is counter to classic microeconomic thinking that assumes returns are mean-reverting.” It makes me think about feedback loops and this passage from More Than You Know: “Negative feedback is a stabilizing factor, while positive feedback promotes change. Too much of either type of feedback can leave a system out of balance.” Positive feedback loops are seemingly the force that drives conditions away from the mean. How can we think about feedback loops in a more constructive way and are there steps that we can take to understand when/where/how they will appear?  As a follow-up, is there a good mental model for thinking about when and how breakpoints appear in feedback loops?

Michael: There’s been a great deal written about this idea, albeit not necessarily using this exact language. One classic work on this is Everett Rogers’s book, Diffusion of Innovations. He was one of the first to describe how innovations—whether a new seed of corn or an idea—spread. From this a lot of other ideas emanated, including the idea of a tipping point, where momentum for diffusion accelerates. 

The Polya urn model is also useful in this context. A basic version of the model starts with balls of two colors, say black and white, in an urn at some ratio. You then randomly select one ball, match it with a ball of the same color, and replace it. For example, say you started with 3 black balls and 3 white balls, so 50 percent of the balls are black. Now you draw a ball, observe that it’s black, and return it to the urn with an additional black ball. So the percentage of black balls is now 57 percent (4/7). 

This urn model is very simple but demonstrates the principles behind positive feedback nicely. Specifically, it’s nearly impossible in advance to predict what’s going to happen, but once one color gets ahead sufficiently, it dominates the outcomes. (You can play a little more sophisticated version here.) It’s interesting to hit the simulator over and over to simply observe how the outcomes vary.

Another area where this model pops up is in format, or standard, wars. The classic example is Betamax versus VHS, but there are plenty of examples throughout history. Here again, as one standard gets ahead, positive feedback often kicks in and it wins the war.   

Now I don’t think there’s any easy way to model positive feedback, but these are some of the mental models that may help one consider what’s going on.

Elliot: You talk about Munger’s advice to think backwards and invert. I think your first book was Expectations Investing which provided a framework for estimating the embedded assumptions in an equity’s price. Yet you also warn that this way of thinking isn’t for everyone. Was this something you realized after sharing the ideas with many or were you always aware of this? Do you have any ideas for why this has a relatively narrow audience? Is there a natural tie-in to the behavioral biases of humans and why this doesn’t work for everyone? (For example, the human proclivity towards the narrative bias to explain past events) And if so, how can we think backwards more rationally and overcome these biases?

Michael: Steven Crist, the well-known handicapper, has a line about horse race bettors in his essay, “Crist on Value,” that I love to repeat. He says, “The issue is not which horse in the race is the most likely winner, but which horse or horses are offering odds that exceed their actual chances of victory. This may sound elementary, and many players may think they are following this principle, but few actually do.” Take out the word “horse” and insert the word “stock” and you’ve captured the essence of the problem. 

Our natural tendency is to buy what is doing well and to sell what is doing poorly. But as Crist emphasizes, it doesn’t really matter how fast the horse will run, it matters how fast the horse will run relative to the odds on the tote board. Great investors separate the fundamentals from the expectations, and average investors don’t. Most of us are average investors.   

My advice, then, is to try to be very explicit about segregating the fundamentals and the expectations. Sometimes high expectations stocks are attractive because the company will do better still than what’s in the price. Great. That’s a buy. Sometimes there are stocks with low expectations that are dear because the company can’t even meet those beat down results. That’s called a value trap. So, constantly and diligently ask and answer the question, “what’s priced in?” Doing so is very helpful.  

Learning Risk and the "Limits to Forecasting and Prediction" With the Santa Fe Institute

Last October, I had the privilege to attend Santa Fe Institute and Morgan Stanley's Risk Conference, and it was one of my most inspiring learning experiences of the year (read last year's post on the conference, and separately, my writeup of Ed Thorp's talk about the Kelly Criterion). It's hard not to marvel at the brainpower concentrated in a room with some of the best practitioners from a variety of multi-disciplinary fields ranging from finance to physics to computer science and beyond and I would like to thank Casey Cox and Chris Wood for inviting me to these special events.  

I first learned about the Santa Fe Institute (SFI) from Justin Fox's The Myth of the Rational Market. Fox concludes his historical narrative of economics and the role the efficient market hypothesis played in leading the field astray with a note of optimism about the SFI's application of physics to financial markets. Fox highlights the initial resistance of economists to the idea of physics-based models (including Paul Krugman's lament about "Santa Fe Syndrome") before explaining how the profession has in fact taken a tangible shift towards thinking about markets in a complex, adaptive way.  As Fox explains:

These models tend to be populated by rational but half-informed actors who make flawed decisions, but are capable of learning and adapting. The result is a market that never settles down into a calmly perfect equilibrium, but is constantly seeking and changing and occasionally going bonkers. To name just a few such market models...: "adaptive rational equilibrium," "efficient learning," "adaptive markets hypothesis," "rational belief equilibria." That, and Bill Sharpe now runs agent-based market simulations...to see how they play out.

The fact that Bill Sharpe has evolved to a dynamic, in contrast to equilibrium-based perspective on markets and that now Morgan Stanley hosts a conference in conjunction with SFI is telling as to how far this amazing multi-disciplinary organization has pushed the field of economics (and importantly, SFI's contributions extend well beyond the domain of economics to areas including anthropology, biology, linguistics, data analytics, and much more). 

Last year's focus on behavioral economics provided a nice foundation upon which to learn about the "limits to forecasting and prediction." The conference once again commenced with John Rundle, a physics professor at UC-Davis with a specialty in earthquake prediction, speaking about some successful and some wrong natural disaster forecasts (Rundle operates a great site called OpenHazards). Rundle first offered a distinction between forecasting and prediction. Whereas prediction is a statement validated by a single observation, forecasting is a statement for which multiple observations are required for a confidence level.

He then offered a permutation of risk into its two subcomponents. Risk = Hazard x exposure.  The hazard component relates to your forecast (ie the potential for being wrong) while the exposure relates to the magnitude of your risk (ie how much you stand to lose should your forecast be wrong). I find this a particularly meaningful breakdown considering how many colloquially conflate hazard with risk, while ignoring the multiplier effect of exposure.

As I did last year, I'll share my notes from the presentations below. Again, I want to make clear that my notes are geared towards my practical needs and are not meant as a comprehensive summation of each presentation. I will also look to do a second post which sums up some of the questions and thoughts that have been inspired by my attendance at the conference, for the truly great learning experiences tend to raise even more questions than they do offer answers.

Antti Ilmanen, AQR Capital

With Forecasting, Strategic Beats Tactical, and Many Beats Few

Small, but persistent edges can be magnified by diversification (and to a lesser extent, time). The bad news is that near-term predictability is limited (and humility is needed) and long-term forecasts which are right might not setup for good trades. I interpret this to mean that the short-term is the domain of randomness, while in the long-term even when we can make an accurate prediction, the market most likely has priced this in.

Intuitive predictions inherently take longer time-frames. Further, there is performance decay whereby good strategies fade over time. In order to properly diversify, investors must combine some degree of leverage with shorting. Ilmanen likes to combine momentum and contrarian strategies, and prefers forecasting cross-sectional trades rather than directional ones.

When we make long-term forecasts for financial markets, we have three main anchors upon which to build: history, theory, and, current conditions. For history, we can use average returns over time, for theory, we can use CAPM, and for current conditions we can apply the DDM. Such forecasts are as much art as they are science and the relative weights of each input depend on your time-horizon (ie the longer your timeframe, the less current conditions matter for the inevitable accuracy of your forecast).

Historically the Equity Risk Premium (ERP) has averaged approximately 5%, and today's environment the inverse Schiller CAPE (aka the cyclically adjusted earnings yield) is approximately 5%, meaning that 4-5% long run returns in equity markets are justifiable, though ERPs have varied over time. Another way to look at projected returns is through the expected return of a 60/40 (60% equities / 40% bonds) portfolio. This is Ilmanen's preferred methodology and in today's low-rate environment the prospects are for a 2.6% long-run return.

In forecasting and market positioning, "strategic beats tactical." People are attracted to contrarian signals, though the reality of contrarian forecasting is disappointing. The key is to try and get the long-term right, while humbly approaching the tactical part of it. Value signals like the CAPE tend to be very useful for forecasting. To highlight this, Ilmanen shared a chart of the 1/CAPE vs. the next five year real return.

Market timing strategies have "sucked" in recent decades. In equity, bond and commodity markets alike, Sharpe Ratios have been negative for timing strategies. In contrast, value + momentum strategies have exhibited success in timing US equities in particular, though most of the returns happened early in the sample and were driven more by the momentum coefficient than value. Cheap starting valuations have resulted in better long-run returns due to the dual forces of yield capture (getting the earnings yield) and mean reversion (value reverting to longer-term averages). 

Since the 1980s, trend-following strategies have exhibited positive long-run returns. Such strategies work best over 1-12 month periods, but not longer. Cliff Asness of AQR says one of the biggest problems with momentum strategies is how people don't embrace them until too late in each investment cycle, at which point they are least likely to succeed. However, even in down market cycles, momentum strategies provided better tail-risk protection than did other theoretically safe assets like gold or Treasuries.  This was true in eight of the past 10 "tail-risk periods," including the Great Recession.

In an ode to diversification, Ilmanen suggested that investors "harvest many premia you believe in," including alternative asset classes and traditional capital markets. Stocks, bonds and commodities exhibit similar Sharpe Ratios over long time-frames, and thus equal-weighting an allocation to each asset class would result in a higher Sharpe than the average of the constituent parts. We can take this one step farther and diversify amongst strategies, in addition to asset classes, with the four main strategies being value, momentum, carry (aka high yield) and defensive.

Over the long-run, low beta strategies in equities have exhibited high returns, though at the moment low betas appear historically expensive relative to normal times.  That being said, value as a signal has not been useful historically in market-timing.

If there are some strategies that exhibit persistently better returns, why don't all investors use them? Ilmanen highlighted the "4 c's" of conviction, constraints, conventionality and capacity as reasons for opting out of successful investment paths.

Henry Kaufman, Henry Kaufman & Company

The Forecasting Frenzy

Forecasting is a long-term human endeavor, and the forecaster in the business/economics arena is from the same vein as soothsayers and palm readers. In recent years, the number of forecasters and forecasts alike has grown tremendously. Sadly, forecasting continues to fail due to the following four behavioral biases:

1. Herding--forecasts minimally fluctuate around a mean, and few are ever able to anticipate dramatic changes. When too many do anticipate dramatic changes, the path itself can change preventing such predictions from coming true.
2. Historical bias--forecasts rest on the assumption that the future will look like the past. While economies and markets have exhibited broad repetitive patterns, history "rhymes, but does not repeat."
3. Bias against bad news--No one institutionally predicts negative events, as optimism is a key biological mechanism for survival. Plus, negative predictions are often hard to act upon. When Kaufman warned of interest rate spikes and inflation in the 1970s, people chose to tune him out rather than embrace the uncomfortable reality. 
4. Growth bias--stakeholders in all arenas want continued expansion and growth at all times, even when it is impractical.

Collectively, the frenzy of forecasts has far outpaced our ability to forecast. With long-term forecasting, there is no scientific process for making such predictions. An attempt to project future geopolitical events based on the past is a futile exercise. In economics, fashions contribute to unsustainable momentums, both up and down, that lead to considerable challenges in producing accurate forecasts.

Right now, Kaufman sees some worrying trends in finance. First, is the politicization of monetary policy, and he fears this will not reverse soon. The tactics the Fed is undertaking today are unprecedented and becoming entrenched. The idea of forward guidance in particular is very dangerous, for they rely entirely upon forecasts. Since it's well established that even expert forecasts are often wrong, then logic dictates that the entire concept of forward guidance is premised on a shaky foundation. Second, monetary policy has eclipsed fiscal policy as our go-to remedy for economic troubles. This is so because people like the quick and easy fixes offered by monetary solutions, as opposed to the much slower fiscal ones. In reality, the two (fiscal and monetary policy) should be coordinated. Third, economists are not paying enough attention to increasing financial concentration. There are fewer key financial institutions, and each is bigger than what used to be regarded as big. If/when the next one fails, and the government runs it through the wind-down process, those assets will end up in the hands of the next remaining survivors, further concentrating the industry.

The economics profession should simply focus on whether we as a society will have more or less freedom going forward. Too much of the profession instead focuses on what the next datapoint will be. In the grand scheme of things, the next datapoint is completely irrelevant, especially when the "next" completely ignores any revisions to prior data.  There is really no functional, or useful purpose for this type of activity.

Bruce Bueno de Mesquita, New York University

The Predictioneer's Game

The standard approach to making predictions or designing policy around questions on the future is to "ask the expert." Experts today are simply just dressed up oracles. They know facts, history and details, but forecasts require insight and methods that experts simply don't have. The accuracy of experts is no better than throwing darts. 

Good predictions should use logic and evidence, and a better way to do this is using game theory. This works because people are rationally self-interested, have values and beliefs, and face constraints. Experts simply cannot analyze emotions or account for skills and clout in answering tough geopolitical questions. That being said, game theory is not a substitute for good judgment and it cannot replace good internal debate.

People in positions of power have influencers (like a president and his/her cabinet). In a situation with 10 influencers, there are 3.6 million possible interactions that exist in a complex adaptive situation (meaning what one person says can change what another thinks and does). In any single game, there are 16 x (N^2-N) possible predictions, where N is the number of players.

In order to build a model that can make informed predictions, you need to know who the key influencers are. Once you know this, you must then figure out: 1) what they want on the issue; 2) how focused they are on that particular problem; 3) how influential each player could be, and to what degree they will exert that influence; and, 4) how resolved each player is to find an answer to the problem.  Once this information is gathered, you can build a model that can predict with a high degree of accuracy what people will do.  To make good predictions, contrary to what many say, you do not need to know history. It is much like a chessmaster who can walk up to a board in the middle of a game and still know what to do next.

With this information, people can make better, more accurate predictions on identified issues, while also gaining a better grasp for timing. This can help people in a game-theory situation come up with strategies to overcome impediments in order to reach desired objectives.

Bueno de Mesquita then shared the following current predictions:

• Senkaku Island dispute between China and Japan - As a relevant aside, Xi Jinping's power will shrink over the next three years. Japan should let their claims rest for now, rather than push. It will take two years to find a resolution, which will most likely include a joint venture between Japan and China for expropriation of the natural gas reserves.
• Argentina - The "improvements" in today's business behavior are merely aesthetic in advance of the key mid-term elections. Kirshner is marginalizing political rivals, and could make a serious move to consolidate power for the long-term.
• Mexico - There is a 55% chance of a Constitutional amendment to open up energy, a 10% chance of no reform, and a 35% chance for international oil companies to get deep water drilling rights.  Mexico is likely to push through reforms in fiscal policy, social security, energy, labor and education, and looks to have a constructive backdrop for economic growth.
• Syria with or without Assad will be hostile to the Western world.
• China will look increasingly inward, with modest liberalization on local levels of governance and a strengthening Yuan.
• The Eurozone will have an improving Spain and a higher likelihood that the Euro currency will be here to last.
• Egypt is on the path to autocracy.
• South Africa is at risk of turning into a rigged autocracy.

Aaron Clauset, University of Colorado and SFI

Challenges of Forecasting with Fat-Tailed Data

(Please note: statistics is most definitely not my strong suit. The content in Clauset's talk was very interesting, though some of it was over my head. I will therefore try my best to summarize the substance based on my understanding of it)

In attempting to predict fat-tail events, we are essentially trying to "predict the unpredictable." Fat tails exhibit high variance, so the average of a sample of data does not represent what is seen numerically. In such samples, there is a substantial gap between the two extremes of the data, and we see these distributions in book sales (best-sellers like Harry Potter), earthquakes (power law distributions), market crashes, terror attacks and wars. With earthquakes, we know a lot about the physics behind them, and how they are distributed, whereas with war we know it follows some statistical pattern, but the data is dynamic instead of fixed. This is true with war, because certain events influence subsequent events, etc.

Clauset approached the question of modeling rare events through an attempt to ascertain how probable 9/11 was, and how likely another one is. The two sides of answering this question are building a model (to discover how probable it was) and making a prediction (to forcast how likely another would be). For the purposes of the model, one would care only about large events because they have disproportionate consequences. When analyzing the data, we don't know what the distribution of the upper tail will look like because there simply are not enough datapoints. In order to overcome these problems, the modeler needs to separate the tail from the body, build a multiple tail model, bootstrap the data and repeat.

In Clauset's analysis of the likelihood for 9/11, he found that it was not an outlier based on both the model, and the prediction. There is a greater than 1% chance of such an event happening. While this may sound small, it is within the realm of possible outcomes, and as such it deserves some attention. This has implications for policymakers, because considering it is a statistical possibility, we should pursue our response within a context that acknowledges this reality.

There are some caveats to this model however. An important one is that terrorism is not a stationary process, and events can create feedback loops which drive ensuing events. Further, events themselves that in the data appear independent are not actually so. When forecasting fat tails, model uncertainty is always a big problem. Statistical uncertainty is a second one, due to the lack of enough data points and the large fluctuations in the tails themselves. Yet still, there is useful information within the fat tails which can inform our understanding of them. 

Philip Tetlock, University of Pennsylvania

Geopolitical Forecasting Tournaments Test the Limits of Judgment and Stretch the Boundaries of Science

I summarized Tetlock's talk at last year's SFI Risk Conference, so I suggest checking out those notes on the IARPA Forecasting Tournament as well. IARPA has several goals/benefits: 1) making explicit one's implicit theories of good judgment; 2) getting people in the habit of treating beliefs like testable hypothesis; and, 3) helping people discover the drivers of probabilistic accuracy. (All of the above are reasons I would love to participate in the next round). With regard to each area there are important lessons. 

There is a spectrum that runs from perfectly predictable on the left to perfectly unpredictable on the right, and no person or system can perfectly predict everything. In any prediction, there is a trade-off between false positives and correct hits. This is called the accuracy function. 

With the forecasting tournament, people get to put their pet theories to the test. This can help improve the "assertion-to-evidence" ratios in debates between opposing schools of thought (for example, the Keynesians vs the Hayekians). Predictions would be a great way to hold opposing schools of thought accountable to their predictions, while also eliciting evidence as to why events are expected to transpire in a given way.

In the tournament, the participants are judged using a Brier Score, a measure that originated in weather forecasting to determine accuracy on probabilistic predictions over time. The people who perform best tend to have a persistence in good performance. The top 2% of performers from one year demonstrated minimal regression to the mean, leading to the conclusion that predictions are 60% skill and 40% luck on the luck/skill spectrum.

There are tangible benefits of interaction and collaboration. The groups with the smartest, most open-minded participants consistently outperformed all others. Those who used probabilistic reasoning in making predictions were amongst the best performers. IARPA concentrated the talent of some of the best performers in order to see if these "super teams" could beat the "wisdom of crowds." Super teams did win quite handily. Ability homogeneity, rather than a problem, was an enhancer of successes. Elitist algorithms were used to generate forecasts by "extremizing" the forecasts from the best forecasters, and weighting those most heavily (5 people with a .7 Brier would upgrade to approximate a .85 based on the non-correlation of their success. Slight digression: it was interesting sitting behind Ilmanen during this lecture and seeing him nod his head, as this theme resonated perfectly with his points on diversifaction in a portfolio resulting in the portfolio's Sharpe Ratio being above the average of its constituent parts). 

There are three challenges when thinking about the value of a forecasting tournament. First, automation from machines is getting better, so why bother with people? While this is important, human judgment is still a very valuable tool and can actually improve the performance of these algorithms. Second, the efficient market theory argues that what can be anticipated is already "priced in" so there should be little economic value to a good prediction anyway. Yet markets and people alike have very poor peripheral vision and good prediction can in fact be valuable in that context. Last, game theory models like Buena de Mesquita's can distill inputs from their own framework. While this may be a challenge, it's probably even better as a complementary endeavor.

How did Ed Thorp Win in Blackjack and the Stock Market?

My earlier post laid out some important lessons on behavioral economics learned from Santa Fe Institute’s conference on Risk: the Human Factor.  The specific lecture that first caught my eye when I saw the roster was Edward Thorp’s discussion on the Kelly Capital Growth Criterion for Risk Control.  I had read the book Fortune’s Formula and was fascinated by one of the core concepts of the book: the Kelly Criterion for capital appreciation. Over time, I have incorporated Kelly into my position-sizing criteria, and was deeply interested in learning from the first man who deployed Kelly in investing.  It's been mentioned that both Warren Buffett and Charlie Munger discussed Kelly with Thorp and used it in their own investment process.  Thus, I felt it necessary to give this particular lecture more attention.

In its simplest form, the Kelly Criterion is stated as follows:

The optimal Kelly wager = (p*(b+1)—1) / b where p is the probability (% chance of an event happening) and b is the odds received upon winning ($b per every $1 wagered).

It was Ed Thorp who first applied the Kelly Criterion in blackjack and then in the stock market.  The following is what I learned from his presentation at SFI. 

Thorp had figured out a strategy for counting cards, but was left wondering how to optimally manage his wager (in investing parlance, we’d call this position sizing).  The goal was a betting approach which would allow for the strategy to be deployed over a long period of time, for a maximized payout.  With the card counting strategy, Thorp in essence was creating a biased coin (a coin toss is your prototypical 50/50 wager, however in a biased coin, the odds are skewed to one side).  This question was approached from a position of how does one deal with risk, rationally?  Finding such a rational risk management strategy was very important, because even with a great strategy in the casino, it was all too easy to go broke before ever attaining successful results.  In other words, if the bets were too big, you would go broke fast, and if the bets were too small you simply would not optimize the payout.

Thorp was introduced to the Kelly formula by his colleague Claude Shannon at MIT.  Shannon was one of the sharpest minds at Bell Labs prior to his stint at MIT and is perhaps best known for his role in discovering/creating/inventing information theory.  While Shannon was at Bell Labs, he worked with a man named John Kelly who wrote a paper called “New Interpretation of Information Rate.”  This paper sought a solution to the problem of a horse racing gambler who receives tips over a noisy phone line.  The gambler can’t quite figure out with complete precision what is said over the fuzzy line; however, he knows enough to make an informed guess, thus imperfectly rigging the odds in his favor. 

What John Kelly did was figure out a way that such a gambler could bet to maximize the exponential rate of the growth of capital.  Kelly observed that in a coin toss, the bet should be equal to one’s edge, and further, as you increase your amount of capital, the rate of growth inevitably declines.

Shannon showed this paper to Thorp presented with a similar problem in blackjack, and Thorp then identified several key features of Kelly (g=growth below):

1. If G>0 then the fortune tends towards infinity.
2. If G<0 then the fortune tends towards 0.
3. If g=0 then Xn oscillates wildly.
4. If another strategy is “essentially different’ then the ratio of Kelly to the different strategy tends towards infinity.
5. Kelly is the single quickest path to an aggregate goal.

This chart illustrates the points:

The peak in the middle is the Kelly point, where the optimized wager is situated.  The area to the right of the peak, where the tail heads straight down is in the zone of over-betting, and interestingly, the area to the left of the Kelly peak corresponds directly to the efficient frontier. 

Betting at the Kelly peak yields substantial drawdowns and wild upswings, and as a result is quite volatile on its path to capital appreciation.  Therefore, in essence, the efficient frontier is a path towards making Kelly wagers, while trading some portion of return for lower variance.  As Thorp observed, if you cut your Kelly wager in half, then you can get 3/4s the growth with far less volatility. 

Thorp told the tale of his early endeavors in casinos, and how the casinos scoffed at the notion that he could beat them.  One of the most interesting parts to me was how he felt emotionally despite having confidence in his mathematical edge. Specifically, Thorp felt that the impact of losses placed a heavy psychological burden on his morale, while gains did not have an equal and opposite boost to his psyche.  Further, he said that he found himself stashing some chips in his pocket so as to avoid letting the casino see them (despite the casino having an idea of how many he had outstanding) and possibly as a way to prevent over-betting.  This is somewhat irrational behavior amidst the quest for rational risk management

As the book Bringing Down the House and the movie 21 memorialized, we all know how well Kelly worked in the gambling context.  But how about when it comes to investing?  In 1974, Thorp started a hedge fund called Princeton/Newport Partners, and deployed the Kelly Criterion on a series of non-correlated wagers. To do this, he used warrants and derivatives in situations where they had deviated from the underlying security’s value.  Each wager was an independent wager, and all other exposures, like betas, currencies and interest rates were hedged to market neutrality. 

Princeton/Newport earned 15.8% annualized over its lifetime, with a 4.3% standard deviation, while the market earned 10.1% annualized with a 17.3% standard deviation (both numbers adjusted for dividends).  The returns were great on an absolute basis, but phenomenal on a risk-adjusted basis.  Over its 230 months of operation, money was made in 227 months, and lost in only 3.  All along, one of Thorp’s primary concerns had been what would happen to performance in an extreme event, yet in the 1987 Crash performance continued apace. 

Thorp spent a little bit of time talking about the team from Long Term Capital Management and described their strategy as the anti-Kelly.  The problem with LTCM, per Thorp, was that the LTCM crew “thought Kelly made no sense.”  The LTCM strategy was based on mean reversion, not capital growth, and most importantly, while Kelly was able to generate returns using no leverage, LTCM was “levering up substantially in order to pick up nickels in front of a bulldozer.”

Towards the end of his talk, Thorp told the story of a young Duke student who read his book called Beat the Dealer, about how to deploy Kelly and make money in the casino.  This young Duke student then ventured out to Las Vegas and made a substantial amount of money.  He then read Thorp’s book Beat the Market and went to UC-Irvine, where he used the Kelly formula in convertible debt to again make good money. Ultimately this young built the world’s largest bond fund—Pacific Investment Management Company (PIMCO).  This man was none other than Bill Gross and Thorp drew the important connection between Gross’ risk management as a money manager and his days in the casino.

During the Q&A, Bill Miller, of Legg Mason fame, asked Thorp an interesting two part question: is it more difficult to get an edge in today’s market? And Did LTCM not know tail risk and/or realize the correlations of their bets?  Thorp said that today the market is no more or less difficult than in year’s past.  As for LTCM, Thorp argued that their largest mistake was in failing to recognize that history was not a good boundary (plus the history LTCM looked at was only post-Depression, not age-old) and that without leverage, LTCM did not have a real edge. This is key—LTCM was merely a strategy to deploy leverage, not one to get an edge in the market.

I had the opportunity to ask Thorp a question and I wanted to focus on the emotional element he referenced from the casino days.  My question was:  upon recognizing the force of emotion upon himself, how did he manage to overcome his human emotional impediments and place complete conviction in his formula and strategy?  His answer was a direct reference to Daniel Kahneman’s Thinking, Fast and Slow, whereby he used his system 2, the slow thinking system, in order to force himself to follow the rules outlined by his formulas and process.  Emotion was a human reaction, but there was no room to afford it the opportunity to hinder the powerful force that is mathematics.