Let's Review: Seeing What Others Don't

I picked up Gary Klein’s Seeing What Others Don’t after reading this line in his paper on the Data-Frame theory of sensemaking:

Successful sensemaking achieves a mental balance by fitting data into a well-framed relationship with other data. This balance will be temporary because dynamic conditions continually alter the landscape. Nevertheless, the balance, when achieved, is emotionally satisfying in itself.

I love the feeling when an insight appears… so much so that I’m skeptical of it. We’ve all seen or perpetrated the crime of running away with something that turns out to be incorrect. And I worry that I ignore obvious solutions because I’m chasing that insight high. I hoped that Dr. Klein’s research offered guardrails (or, in his terms, new frames and anchors) so that I could learn when I should and shouldn’t trust my own “a-ha!” moments.

Seeing What Others Don’t is a book about insights: what they are, how we arrive at them, and how we prevent or encourage ourselves from finding them. It started out of pure curiosity. For years, Dr. Klein collected stories about moments of insight. At some point, he set aside some spare time to apply his naturalistic research methods¹ to this pile of 120 cases, and eventually he decided to compile what he learned into a book. It has three sections, which are: 1) what insight actually is 2) what interferes with insights 3) ideas for fostering insight.

What, Like, Even Is Insight, Man?

The first section is the strongest part of the book, where most of the meat lies. Klein starts by presenting this model for performance improvement, improvement = reducing errors + increasing insights.

Mostly, we seek to reduce errors because that is more controllable. So how do we “boost the up arrow?” This question sparked the curiosity that led to Klein collecting examples of insights.

The book presents 5 stories of insights up front to ground the investigation. (The entire book is filled with stories: there are 60 in total, listed in a dedicated index at the back of the book). Klein reviews the existing literature on insight, testing it against these examples. The first modern theory of insight came from Graham Wallas in 1926. He contended that insight occurs in 4 stages: preparation, incubation, illumination, and verification. Klein uses a couple cases to point out problems with this model, including that “preparation” looks more like “being an expert in the domain” rather than “doing a bunch of task specific research then going for a walk”².

Klein ends up defining an insight as “an unexpected transition from a mediocre story to a better one”. They are “coherent and unambiguous” and “inspire confidence”. He then details his methodology for finding and reviewing such stories, showing his work to build confidence in how he reached his conclusions.

The four categories of sources of insights that Klein discovered are as follows (I’ve included an example of each one):

1. Connections: In WW2, Japanese Admiral Yamamoto realized that a novel British airborne torpedo attack on the Italian fleet meant that Pearl Harbor was vulnerable to the same style of attack.
2. Coincidences and curiosities: coincidences are patterns, such as Dr. Michael Gottlieb noticing a few cases of immune system issues within the gay community in Los Angeles in the 1980s, leading to the discovery of AIDS. Curiosities are one-off events, such as when Alexander Fleming discovered dead staph bacteria in a mold contaminated sample and went onto create penicillin, the world’s first antibiotic.
3. Contradictions: The discovery of cholera propagating via dirty water came, in part, because someone noticed that the victims had damage in the digestive systems. Miasma theory, the explanatory model of the day, predicted that the lungs should have shown damage.
4. Creative Desperation: In 1949, Wagner Dodge survived a wildfire chasing him and his team up a hill by setting fire to the grass in front of him, depriving the wildfire of the fuel it needed to burn him.³

After coding each story along these dimensions, Klein synthesized a descriptive Triple Path Model of insight:

“Anchor” will look familiar to anyone who’s read Klein’s data-frame work. It’s a big part of how he believes that we make sense of the world. He notes that this Triple Path Model is overly simplistic, and will probably be changed by future research. But the punchline is compelling:

The Triple Path Model shows why earlier accounts aren’t wrong as much as they are incomplete. They restrict themselves to a single path. Researchers and theorists such as Wallas who describe insight as escaping from fixation and impasses are referring to the creative desperation path. Researchers who emphasize seeing associations and combinations of ideas are referring to the connection path. Researchers who describe insight as reformulating the problem or restructuring how people think have gravitated to the contradiction path. None of them are wrong. The Triple Path Model of insight illustrates why people seem to be talking past each other. It’s because they’re on different paths.

How to Prevent Insights

We fail to achieve insights due to stupidity. The book defines this as failing to notice a connection or contradiction, despite having all the required information. Klein found 30 cases in his sample where multiple people had enough information but only one gained the insight. He calls these “twins”. Studying these revealed four sources of stupidity (again, I’ve added an example of each).

• Flawed beliefs: In 1962, a CIA member ignored data suggesting Russians were moving nuclear warheads to Cuba because they didn’t believe the Russians would be willing to risk nuclear conflict. Two thirds of the 30 twins cases had examples of flawed beliefs.
• Lack of experience: Napoleon’s military career took off via an insight that won the Siege of Toulon based on his experience as an artillery officer that, experience that his commanding officer just did not have. Two thirds of insights in the entire 120 case set depend on having experience.
• A passive stance: Klein’s daughter Devorah moved up 2 spots in their fantasy baseball league on the final day of her first season because she found a loophole in the way pitchers’ stats counted on that last day. Everyone else in the league was content with their grasp of the league rules, so they stopped looking for new data. Again, two thirds of the 30 twins cases relied on having an active stance.
• Concrete reasoning style: Half of insights in the 30 twins cases had one person who was more comfortable with ambiguity and contradictions. Klein mentions a few cases, but the full opening quote of this section is worth a read⁴:

People differ in how well they tolerate contradictions and ambiguity, and this personality style likely affects their success at gaining insights. People also differ in how ready they are to entertain ideas that they don’t think are true and in how much they enjoy imagining alternative universes. Some people become impatient with speculation. They see the playful exploration of ideas as a sign of immaturity. They want closure, and they roll their eyes when a member of the group starts going off on tangents. They are concrete thinkers who just want to work with the facts, not with flights of fancy. This concrete reasoning style wouldn’t leave people very open to insights.

The playful reasoning style likes to juggle ideas and imagine hypothetical scenarios.

The book uses the discovery of DNA by James Watson and Francis Crick to run through all four sources of stupidity.

One thing that stood out to me was that, despite being generally inexperienced as researchers, they had more experience in biochemistry than the average geneticist, which allowed them to challenge certain flawed beliefs. Klein’s research is somewhat unique in the psychology field because it shows that expertise is usually a good thing. This runs counter to the “heuristic and biases” school of thought that is prevalent in the field⁵. But the book makes that argument with nuance. Experience can trap someone into flawed beliefs or a passive stance. The Watson and Crick story adds another caveat: sometimes the domain changes and even a small amount of experience in a newly relevant area will create an overall experience advantage. Klein points out that there’s a huge amount of luck involved as well. Watson and Crick had really fortunate timing.

The rest of the section analyzes three domains: software design, organizational design and the experimental design of insight research.

Klein argues that software designed with best practices should filter out irrelevant data, and that this necessarily leads to weaker insights. As someone who has been casually interested in joint cognitive systems, this line bummed me out, because the the book doesn’t really offer a path out of it:

Therefore, I am not optimistic about the chances for designing systems that foster insight. The four guidelines at the beginning of this chapter are too compelling. In the past, when I led seminars on cognitive systems engineering, I advocated for these guidelines. I was a believer until I started my project to investigate insight.

In a similar way, organizations that follow best practices value predictability and perfection (the absence of errors). Insights are unpredictable and often look like errors. Managers tend to discourage this in their reports; committees tend to sand down the rough edges that might generate insights; people frequently self censor.

Remember how Klein is known for having a positive view toward experience? Well, he’s not a fan of how insight research treats expertise. Many insight problems are “gotcha” problems, where experience is actually weaponized against the subject. One chapter starts with a bulleted list of great ways to suppress insights, before boom-roasting the rest of the psychology field by saying that this looks like the description of the experimental design for research on insight.

It’s a bit inside baseball-y, but Klein’s conclusion is relevant to the layman. Much of the existing insight research focuses on the creative desperation pathway, by creating impasses under time pressure. So the research is incomplete, ignoring two of the three paths to insight in his Triple Path Model. I’ll certainly keep this in mind when hearing conclusions about how “studies show” this or that about creativity and insight.

How to Encourage Insights

I’ll be honest, I expected this section to be more positive than it actually was. We go from learning about how our best practices crush insight to a few half baked recommendations that Klein is doubtful will work. This is not surprising, though, considering that the research ended with section 2. The last section isn’t based on experimental findings. Klein is opining from his (many years of) experience.

The recommendations roughly track back to the sources of insight and impediments:

• Contradictions: pay attention to your “Tilt!” reflex, when you feel something isn’t quite right.
• Connections: Klein is skeptical that “swirl”, or exposing yourself to many new ideas, will be beneficial, as this will also generate a flood of irrelevant data to sift through.
• Flawed Beliefs: Klein talks about how people often list out all of their assumptions in order to correct flawed beliefs, but he has seen no evidence that this is effective. [I should note that I’ve seen real world success from folks listing assumptions as part Goldratt’s evaporating cloud technique]
• Critical Thinking: Defined as “a systematic review of available evidence”, Klein believes this has a place for generating insight, but that we must be careful not to let it make us mindless or dampen the playful thinking style.
• Incubation: There is growing academic evidence that letting ideas sit around for awhile can help generate insights. So the whole “sleep on it” or “go take a walk” thing has some merit.

If you’re keeping track at home, 3 of the 5 recommendations are caveated with “I doubt this will actually work”. Great!

There’s another chapter on advice for guiding people towards an insight. This requires that you are in a teaching position and that you can discover what the missing insight is. This boils down to working to see the other person’s perspective before judging or teaching, good advice that reminds me of Roger Martin’s assertive inquiry. In fact, Klein later calls the willingness to really listen and to keep digging appreciative inquiry.

The section wraps up with some recommendations for fostering insights within an organization. Klein suggests creating a dedicated insights group, akin to a quality control division. This rhymes with the 2000s era recommendation, which I first encountered in The Innovator’s Dilemma, to separate your innovation group from the rest of the org so that you can set the appropriately different expectations for them. I’m not sure how this has worked in practice in the past two decades. Klein further recommends using emotion-driven storytelling to communicate insights throughout an organization.

Perhaps the most interesting part of this entire section is where Klein talks about the organizational willpower required to encourage insights. Since the default mode of an organization is to prevent errors, someone needs to assume the risk of countering that inertia. Klein calls this a willingness to change the goal of the organization itself, and uses the success of the counterinsurgency in Iraq to illustrate it.

A willingness to change the goal reminded me of effectuation, where goals are created on the fly from the means at hand. New businesses are frequently effectuated, and they tend to develop a management mindset (figuring out the means required for a fixed goal) as they mature. This isn’t a really a failure, it’s just a natural tendency for organizations. Big ships turn slowly and so forth. But I’m also reminded me of how Jeff Bezos seemed to fluidly navigate between a management and effectuation mindset as the situation warranted. So maybe there is a path out of this tradeoff! (Welcome to how my mind works, bouncing from one tangential idea to the next… I choose to believe I’m practicing Klein’s playful reasoning). I’m still chewing on this idea. If nothing else, it’s a useful to note that one should expect resistance to insights, even when they are good.

Klein later holds up Six Sigma as an anti-pattern to learn from. According to him, it absolutely crushed innovation in the organizations that adopted it. This surprised me! Early in my career, I worked in a super innovative engineering company that implemented some Six Sigma like practices. Klein has receipts to back up his claim, but I was curious whether this represented the whole story.

In total, I didn’t find many things in the section to be actionable. At the very least, it was interesting to see that the smartest minds are still wrestling with what to do with all this. Honestly, that probably one of the reasons I’m fascinated by it. Solved problems are no fun!

My Takeaways

If you’ve read this far, you may be thinking that my overly long helpfully thorough summary can adequately substitute for reading the book yourself. I’d say this is a mistake. To illustrate, I’ll walk through my biggest takeaway, something I haven’t yet even mentioned:

Skepticism is good, actually.

I’m exaggerating. Skepticism is all over the book, sometimes it’s helpful for insight and sometimes it’s a hindrance. One reason I respect Klein is because he avoids simple, overly reductive opinions. But the book does offer some implicit guidance on how to make skepticism productive.

My own relationship with skepticism is confusing. I’ve had my share of teammates that refuse to entertain new ideas. I’ve participated in (and moderated) brainstorming sessions where a “no bad ideas!!” ground rule was accompanied by a tacit understanding not to actually say anything risky that might look like a bad idea. I have a skeptical and contrarian streak, and, since reading this book, I’ve chatted with several friends whom I consider both creative and productive, noticing a deep skepticism in their approaches as well.

So what makes skepticism productive? Klein has a descriptive answer: skepticism is productive when it triggers an accurate “something doesn’t feel quite right” feeling. This is very common in the contradiction path towards insight; Klein found it in two thirds of such cases. The book doesn’t make this case directly, but I gather that skepticism is harmful when it causes us to cling to a flawed belief or passive stance, or pushes away from a playful reasoning style.

Now, there’s a lot of judgment required to distinguish between “not quite right” and “clinging to false beliefs”, but in my own experience, skepticism becomes harmful when it’s defensive. We need those “no bad ideas” qualifiers when a team implicitly believes that a bad idea will damage their credibility. We cling to false beliefs when we embed them into our self esteem. Both of my productively skeptical friends are good at separating self worth from ideas, within themselves and within others. My working theory is that this creates an environment where their natural curiosity more easily flourishes.

This result, that my biggest takeaway is something weirder than the headline content like the Triple Path Model, is predicted by the book itself. I’m bringing a lifetime of unique context into the book with me. The nagging contradiction between keeping an open mind and being skeptical had me searching for answers, and the book triggered new connections in subsequent conversations with friends. This is exactly how Klein describes the process of insight. I didn’t find easy answers to my question on when to trust the feeling of insight (although I’ve found a few new threads in more recent research to chase down), but this is again part of the deal: insight is not predictable. For now, I will encourage the small voice of skepticism that occasionally arises when that sweet, sweet feeling of insight appears, nurturing its contradictions and avoiding its defensiveness. Klein himself acknowledges how difficult this is:

There’s no simple guidance here. Holding on to a flawed theory can be a mistake, but so can trusting flawed data.

I encourage anyone who’s interested in the science of insight and creativity to give this book a read. The wealth of stories make it an easy read, and I’ve gained a few new frames and anchors to play around with in my thinking. Bring your own experience and see what pops out to you. I’ll bet it will be something different than what I noticed!