Categories
Startup

Nassim Taleb – Black Swan

My Opinion

Highly recommend reading this book. While it took me some time to finish it, it was totally worth it. Taleb shares some highly interesting mental models in regard to probabilities, system theory, knowledge creation, life choices and many more domains.

Reading Recommendation: 9/10


Our world is steered by unknown, improbably events that can’t be forecasted. You can call them unknown unknowns or Black Swans, an expression shaped by Nassim Taleb.

  • These events are outliers. There is no regularity in their occurrence and no past data that gives a hint and prepares us to what happens. They carry an extreme impact. And, funny enough, in hindsight they can be easily explained. (Hindsight Bias)
  • These unknown unknowns happen because they are not supposed to happen. This might sound weird but think about it. When you get to know something, you can prevent it. If you don’t, you can’t. Plus, if you counterparts knows you know, they will react differently. Think about 9/11.

The problem of inductive reasoning is that the same data could confirm a theory and also its exact opposite.

  • There are systematic problems that arise when building knowledge based on empirical observations. In a nutshell, it’s fair to say that we know when we are wrong with a much higher confidence then knowing when we are right.
  • As an example, consider a turkey that is fed every day. With every single feeding he builds up his confidence that he will be fed every single day. One afternoon before Thanksgiving, this believe will be proven wrong.
  • What this example shows is that the turkey observations were in fact harmful. With every day that his confidence rose, so did the actual risk. The turkey’s feeling of safety peaked as the risk was the highest.
  • The same set of data can confirm a theory and also its exact opposite. If you survive another day, it could mean that you are getting closer to being immortal or that you are closer to death.

What often matters when learning about properties is how they behave in extreme situations under severe stress.

  • If you want to get know if you can really count on a friend, you need to look at him under the tests of severe circumstances, not under the regular rosy glow of daily life. Only then you will truly understand his personal ethics and his degree of integrity.
  • This is equally true when it comes to understanding health. Would it be possible to understand health without considering wild diseases and epidemics? At the very least it would significantly harder.
  • Indeed the normal is often irrelevant. Therefore, it sometimes helps to deliberately cause a system to fail to learn how and why it reacts that way.

At the first glance, the interconnectedness of globalisation reduces volatility and creates the impression of stability. At the second glance, it shows the increased fragility: There will be less Black Swans at much severe consequences.

  • Financial institutions have been merging into a smaller number of very large banks. Almost all banks are now interrelated.
  • Instead of several loosely connected financial systems, we now have one gigantic, highly dependant system. When only one part fails the whole system crashes.
  • The increased concentration among banks seems to reduce the likelihood of a financial crisis. If one occurs, however, it will be at global scale with devastating consequences.
  • There is another problem. The rarer the event, the less we know about its odds. It means that we know less and less about the possibility of a crisis.

Living mostly in non-linear, black-swan driven environments where forecasts are difficult, we face a serious expert problem.

  • The researcher Philip Tetlock studied the forecasts of political and economic “experts.” He asked various specialists in these domains about the likelihood of a bunch of political, economic and military events occurring about five years ahead. In sum, he collected 27,000 predictions from almost 300 specialists. When looking at the results it turned out that an “expert” status didn’t matter. There was no difference between a PhD or an undergraduate degree. Interestingly enough, Tetlock noticed that the bigger the reputation of a subject, the worse predictors they were.
  • Part of the problem might be the illusion of familiarity. Just because we spent much time studying something, it doesn’t necessarily mean that we are particularly good at understanding where it’s heading.

Almost no discovery, no technologies of note, came from design and planning – they were just Black Swans.

  • It turns out that top-down planning is often much less relevant than we might expect. History is full of examples of serendipitous discoveries. In fact, it seems like randomness and sheer luck played a surprisingly large role in most of our great discoveries.
  • Penicillin is one example of a serendipitous discovery with massive impact. When Alexander Fleming was cleaning up his laboratory, he noticed that penicillium mold had contaminated one of his old experiments. He thus recognized the antibacterial properties of penicillin, the reason many of us are alive today.
  • Viagra, which effects on our society could be considered significant as well, was meant to be a hypertension drug.
  • The laser with various fields of application nowadays is another prime example of a “solution looking for a problem” type of discovery. When the inventor Charles Townes was asked about his discovery he replied that he was satisfying his desire to split light beams. Consider the effects of laser today: compact disks, eyesight corrections, microsurgery, data storage and retrieval. All totally unforeseen and based on some playful tinkering.
  • But this is not only true for complex or entirely new discoveries. It took 6000 years after the invention of wheels (by, we assume, the Mesopotamians) until somebody came up with the idea of adding wheels to suitcases. Isn’t that astonishing? We had been putting our suitcases on top of a cart with wheels, but nobody thought of putting tiny wheels directly under the suitcase. Technology is only trivial retrospectively–not prospectively.

Our epistemic arrogance lets us overestimate what we know and underestimate uncertainty.

  • The following experiment has been conducted many times with different subject matters and populations. The researchers present a question to each person in the room which answer is a number. They then ask the subjects to estimate a range of values for that number so that they have a 98 percent chance of being right. Although the subjects can literally pick any range that they feel confident with, the intended 2 percent error rate usually turns out to be between 15 percent and 30 percent (depending on the population and the subject matter).

Being a victim to the Survivorship bias, we systematically neglect the importance of silent evidence and the role of luck.

  • When researchers study successful people they often look at their similarities like courage, risk taking, optimism and so on. They assume these traits are what make successful people. If you take, however, silent evidence into consideration and look at the cemetery, you will notice that the graveyard is full of failed persons who shared these traits.
  • One question therefore will always be hard to answer: Were theses people successful because or despite these traits?
  • Between the population of successful millionaires and the failed people on the graveyard, there may be some differences in skills, but what truly separates them is one factor: luck.
  • There is a vicious attribute to the survivorship bias: it is the hardest to notice, when its impact is the largest. The more deadly the risks turn out to be the harder it is to find the silent evidence that is so crucial to take into consideration.
  • To avoid the survivorship bias, choosing the right reference point is crucial. Take the example of a gambler. When looking at the population of beginning gamblers, it’s almost certain that one of them will make a small fortune. If your reference point therefore is the entire population, there is no problem. But from the reference point of a winner (so without taking the losers into account, which happens all too often) there seems to be something greater going on than sheer luck.
  • Beyond silent evidence there is another factor to take into consideration. It’s the nature of evolution that it only works in the long-term and that its short-time outcomes often misleading and deceptive. It’s therefore often not obvious which traits are really good for you, especially because second order effects are not apparent.

“Cumulative advantage” is a theory that describes how winning now increases your odds of winning again in the future and vice versa.

  • Failure too is cumulative; losers are likely to also lose in the future, even if we don’t consider demoralization as a consequences of failing.
  • The English language provides a good example for this. Zipf’s law is a mechanism that describes how the more you use a word, the less effortful you will find it to use that word again, so you borrow words from your private dictionary in proportion to their past use. This demonstrates why out of the sixty thousand main words in English, only a few hundred a regularly used in writings, and even fewer commonly appear in conversation.
  • There are many more examples:
    • The more people live in a particular city, the more likely a stranger will be to pick that city as his destination.
    • The more people are using a certain platform, the more value it provides for every new user join.
    • The more successful you are in your job, the more opportunities will be provided to you. (see: Dominance hierarchy)
  • The underlying principle is as simple as this: The big get bigger and the small stay small, or get relatively smaller.

If knowing the underlying equation, predicting the outcome is often easy. However, reverse engineering the process, meaning deriving the equation based on the outcome, is often almost impossible.

  • For example, knowing the mathematical rule for a series of number, deriving the subsequent numbers is extremely easy. The reverse, however, is often extremely difficult.
  • The researcher P. C. Wason presented subjects with the three-number sequence 2, 4, 6 and asked them to try to guess the rule generating it. The subject had to present other three-number sequences based on the rule they had in their mind and wanted to test. The experimenter would answer with “yes” or “no” in regard to the consistency with the actual rule. Once the subjects were confident with their rule, they would formulate it. It turns out that the actual rule was simply “numbers in ascending order”. Very few subjects got this right since everybody tried to confirm their rule as opposed to falsifying it. Having their theory in mind, all the subjects were trying to find confirming evidence.
  • Note the similarities of this research with how we make sense of history. We tend to assume that history follows a certain logic and that, in theory, we should be able to forecast it. However, all we see are the events, never the rules, while still trying to derive overarching theories based on this.

We systematically overestimate the effects of both positive and negative future events on our lives. This tendency is called “anticipated utility” by Daniel Kahneman and “affective forecasting” by Dan Gilbert.

  • The problems seems to be that we don’t pay attention to our past experiences and are unable to learn from them.
  • Examples can be found everywhere. We assume the next promotion will change our life. We are afraid that things will never get back to normal after loosing a close relative. We believe if we get to build our dream house, we will be happy forever. And so on.
  • Unfortunately, this is not how we human beings work. Being the survival-driven social animals that we are, we are trained to quickly adapt to new circumstances and constantly developing new goals and desires to strive towards.
  • In scientific terms this characteristic is referred to as hedonic adaptation and describes the process of humans to constantly adapt to the status quo and to not judge our current state in absolute terms but instead to only perceive relative changes.
  • One of the most cited pieces of research in this domain is a study from 1978 where researchers interviewed two very different groups about their happiness – recent winners of the Illinois State Lottery and recent victims of catastrophic accidents, who were now paraplegic or quadriplegic. The participants were asked how much pleasure they derived from everyday activities such as chatting with a friend or laughing at a joke.
  • When the researchers analysed their results, they found that the recent accident victims reported gaining more happiness from these everyday pleasures than the lottery winners. And even though the lottery winners reported more present happiness than the accident victims (4 out of 5 as compared to 2.96) the authors concluded that “the paraplegic rating of present happiness is still above the midpoint of the scale and the accident victims did not appear nearly as unhappy as might have been expected.”

Round-trip fallacy describes the confusion of absence of evidence for evidence of absence.

  • When examining a patient for cancer, the doctor can share the negative results but saying we couldn’t find any evidence of cancer. The acronym used in the medical literature is NED, which stands for No Evidence of Disease. What the doctor can’t say is we found evidence for no cancer. There is no such thing as END, Evidence of No Disease.
  • One example of the round-trip fallacy can be found when looking at the case of mothers’ milk in the 1960s. Doctors looked down at mothers’ milk as something that could be equally well replicated by their laboratories. Unfortunately, they missed the many useful components of mothers’ milk that are crucial for the development of an infant. A simple confusion of absence of evidence of the benefits of mothers’ milk with evidence of absence of the benefits.
  • Those infants who were not breast-fed had an increased risk of a number of health problems, including a higher likelihood of developing certain types of cancer. Furthermore, benefits to mothers who breast-feed were also not taken into consideration, such as a reduction in the risk of breast cancer.
  • What this teaches us once again is that we know with a lot more confidence when we are wrong (i.e. falsification) then when we are right (confirmation).

Categories
Startup

The Startup We Are Working On: Fount

Introduction

We are living through a Cambrian explosion. An explosion of content on the world wide web that is probably only overshadowed in significance by Gutenberg’s printing machine and the following explosion of printed content. In just the last ten years, worldwide data jumped from 2 to 59 zettabytes, Twitter amassed 500m daily tweets, blogs went from niche to mainstream, and the e-book market in the U.S. alone grew from $1B to $6B.

The way we consume content is more diverse than ever. There are blogs and newsletters, Medium and Substack, Twitter threads and tweetstorms, podcasts and Youtube, and much more. By the time you read this who knows what the next big thing will be? While this Cambrian explosion of digital content is beautiful, it’s also overwhelming, unstructured and noisy. But still, when digging through your feed, there are true gems, bits and pieces of insights & information that you just wouldn’t stumble across otherwise. These can change the way you see the world or simply offer an amusing perspective. You can learn from some of the smartest people in the world and go down incredible rabbit holes just by following your curiosity online. Simultaneously, we see more and more people gaining access to this seemingly limitless world by getting online. And they don’t just consume – they create!

The problem is no longer how to access information, it’s what you do once you have it. How many incredible insights are never saved or end up on the bottom of siloed bookmarks, lists or note-taking tools? How many are never truly digested, connected with related bits and pieces and shared with others to spark ideas or inspire them? And how amazing is it, when this does happen? It’s those “happy accidents”, serendipity, driven by curiosity and tinkering, that are special and drive ideas and technology forward.

“Curiosity demands that we ask questions, that we try to put things together and try to understand this multitude of aspects as perhaps resulting from the action of a relatively small number of elemental things and forces in an infinite variety if combinations” – Richard Feynman

Our mission at Fount is to make these “happy accidents” happen more often. Think a 100 times more often. It’s quite selfish, to be honest. We love to read and learn online and don’t have something that scratches this itch. We believe that doing so would be a net positive and could unlock ideas and thoughts across disciplines that change our world.

The Status Quo

The Problem of Silos and Noise

It shouldn’t come as a surprise to anyone after reading the introduction, that we are not quite satisfied with the existing approaches – namely note-taking tools – to manage and expand your knowledge. We believe there is a lot of untapped potential.

First, there are the traditional note-taking tools that allow you to save notes in a linear and static manner. As we speak, one of the largest ones, one that innovated the space ten years ago, is on a feature moratorium for 18 months to fix their backend – not quite the source of innovation today.

Then, we have a new generation of note-takers that bring exciting twists to an old game. Leveraging backlinks or knowledge graphs as well innovation in embedding blocks in pages, they allow for more networked structures and flexibility. Content from the web can usually be saved into these page structures and then the tools differ greatly in what you can do with a page. Yet the common thread between these is that it’s very hard to operate on the insight-level when the tool is built for the note, or page level.

Besides, and this is true for old and new note-taking tools alike, it takes a lot of effort to create and maintain such a system. It’s work, really, and this is one of the largest problems we see. This is not to say that knowledge management can exist without having some discipline in structuring connecting your information properly – but there is a huge potential for making this experience as simple, seamless and playful as possible.

We don’t think that note-taking tools are the sole answer to knowledge management. They do have their purpose. This is, as the name already indicates, allowing the user to put in the effort into deliberately saving and writing down ideas and thoughts and structuring them – depending on the tool – in file-cabinets or as knowledge graphs. But they are not so much about seamlessly saving and structuring insights from all across the web, much less about connecting and digesting them in new and serendipitous-optimised ways. This once again points to the potential we currently see – to make knowledge management about play instead of work.

A New Way of Structuring & Connecting Your Insights Is Needed

The most difficult challenge is probably connecting the dots. How do you structure and connect your insights using existing solutions? Because isn’t that fundamentally how we learn? There really are no stand-alone ideas. Our brains work as associative networks where one idea stands in multiple, interdependent relationships with others.

Even building our internal mental models to make sense of the world around us comes down to making and breaking connections between insights and domains. This is what John Boyd callsa dialectic process of destruction and creation: Using analysis (breaking down a comprehensive whole into its constituents) and synthesis (starting with parts and building towards a comprehensive whole) to approach match-up with observed reality. Arriving at pieces of information or insights is the analysis, the deconstruction of domains. But what tools do we have to create and structure our own concepts of meaning?

Currently, we see two approaches to structuring and connecting knowledge. Firstly, the hierarchical file-cabinet approach and secondly, the knowledge graph approach. While file-cabinets are frequently used, they have some obvious disadvantages. The most crucial one being their lack of interconnectivity. Using a file-cabinet makes it rather difficult to re-use and re-mix distinct insights. If they are duplicated and re-used, one often faces the problem of multiple versions that can cause a lot of confusion.

Knowledge graphs address this problem by enabling a high degree of connectivity. Information can be connected in back-linking, overlapping hierarchies while always being up to date. This approach resembles more the way our human brain works where each piece of knowledge represents a node in a larger network

While we see the potential in knowledge graphs, they are much more difficult to navigate in the world of bits than in the world of neurons. When containing a lot of information, they require complex visualisation and are high maintenance – which can be the right tool for some projects but creates problems when trying to structure insights in a serendipitous, playful way. A best of both worlds solution is currently lacking.

Go With the Flow: The Power of Simplicity & Serendipity

It’s about play

Following your curiosity isn’t work, its play. Genius ideas are rarely stumbled upon during office hours or according to a schedule. Our brains just don’t work that way. Instead, you take a shower or go for a run and suddenly the puzzle pieces fall together, it all starts to make sense.

This state of wandering, free-form contemplation & discovery (or rediscovery) is what we are striving for. The best parts of the internet feel like this and we believe that Nassim Taleb captured it when describing and embodying the lifestyle of a flaneur, somebody who enjoys to wander and to stroll with no other purpose than to be an observer of life in all of its nuances.

Whatever solution you use, it should reflect this basic tendency of how we learn and discover by being seamless and allowing for serendipity. Copying and pasting insights into static pages that require many hours of maintenance is not an adequate solution. Capturing, structuring and connecting your insights, as Feynman puts it, in “an infinite variety of combinations” to create new knowledge should be a fun process.

Insights As Building Blocks

At the centre of our approach is the “insight”. We want to break non-fiction content down to its most fundamental level. An insight could be a tweet you liked, a book passage you highlighted or a quote from an essay that inspired you. But it could also be the podcast clip you listen to on repeat or visual graphics that you don’t want to lose.

Insights form the basic building blocks, the atomic unit, of our product. Their simplicity allows for seamless digesting as well as mixing and remixing of insights. With this fundamental block, you can start building.

Playlists For Simple Value-Adding Structure

What is then needed is an intuitive and seamless possibility to structure and connect your insights. Whenever you read, see or hear something that strikes out, it needs to be easy to add these insights to your library, grouped in theme-based containers that we call ‘playlists’. Imagine you stumble upon a new insight – don’t you usually already have a rough idea on where to store this element?

For the sake of simplicity, we limited the possible number of insights per playlist. Instead of continuing to throw insights into your playlists, you need to consciously select and structure your insights accordingly. The concept here is similar to Twitter, since we believe that limitation will add to the idea of focusing on highly aggregated knowledge and force the user to create value via negativa. If needed, multiple playlists can be grouped to meta-playlists.

Beyond Linearity: Strings For Representing Relationships

Moving beyond the rather simple idea of storing insights in playlists, users can string several insights together, both within and across playlists. The idea here is to move away from file-cabinet and the limitations of hierarchical structures while keeping it as simple as possible to connect insights.

It might be just three or four insights within a playlist, that build upon each other and therefore are stringed together to visualise their relationship. However, it might also be a certain meta-theme across multiple playlists that forms a string. Users can start building strings from just an intuitive, serendipitous feeling, that two or more insights might stand in a causal relationship. If they continue adding to this string, they can at any point choose to convert these strings into an independent playlist.

Bringing It All Together – This Is How You Play…

Whatever insights the user sees or reads that he finds worth saving, he can do so easily via various integrations, both on mobile and desktop.

When structuring and connecting these insights, the concepts of playlists and strings really allow to focus on the state of flow that the user experiences when satisfying their curiosity. It enables the user to take on the conundrum of both the file-cabinet system and the knowledge graph approach to structured information. He can move up and down knowledge structures hierarchically by collecting insights in playlists and sorting playlists into meta-playlists. Or literally “pull on the strings” of good ideas and explore related insights laterally by loosely connecting insights from different contexts with flexible strings.

Automatically created digests-formats as well as manually created playlists allow the user to digest and revisit his knowledge. The automatic digest can be thought of as temporary playlists that contain insights based on time horizon, category or interests (how about getting a time-capsule of your favourite insights from last year?). You can revisit as well as deliberately relearn and connect your insights whenever you want, however you want.

… And This Is How We Do It Together

You can play Fount in single-player mode, solely focused on your own insights. And that is totally fine. But instead of you being the sole curator of your own knowledge, why not utilise the distilled key takeaways, highlights & insights of brilliant minds from around the globe?

Play multi-player by sharing insights and collections of insights with your circle of friends and go through what they are saving, learning & connecting. But we think that there is more to it than that. In our information saturated world, curation is desperately needed. We are looking at Fount as an Insight Curation Platform – a community of curators that share aggregated knowledge and a platform that offers signals rather than noise.

Discover the curations of other users, of friends or knowledge influencers that you follow. Build upon their playlists, import individual insights that you find fascinating or strings of insights that add to your own playlist. Use these insights to create new and exciting connections and trace back the sources to discover even more.

The goal of this Insight Curation Platform has to be to keep it as simple, serendipitous-optimised and seamless as it can get, while allowing for playful curation, structure and connection. Using Fount should feel like one of those conversations where you lose track of space and time, a place for curators and digital flaneurs to create unique knowledge that can be shared with others. Becoming a curator, even making a living from it, then becomes a side effect of increasing the level of serendipity and structured knowledge on the internet.

What’s Next: Tapping Into the Curator Economy

As pointed out by now, good taste is valuable, especially with increasing optionality. If you are curating insights from all across the web – saving, structuring and connecting them – you are adding value. A lot, actually. Why not share this with everybody? And why not use the possibility to tap into the already curated, highly aggregated knowledge from so many other people? Why make it a single-player game when the value of a multi-player game is tenfold?

This is at the core of what we are seeing as the emerging Curator Economy. Li Jin (ex a16z) prominently described the advent of what she framed as the Passion Economy, of more and more people being able to monetise their interests and become creators, not just consumers of content & information.

The Curator Economy can be seen as a subset of this trend, by making it easier to follow one’s curiosity and enabling curators to earn by doing what they love. But it also addresses one of the challenges of the Passion Economy: with more content from more and more decentralised sources, how do you know what to read? Whom to follow? How do you pick out the gems?

Imagine being able to read the distilled key takeaways, highlights & insights of brilliant minds from all across the globe. And as one of those brilliant minds, imagine to curate & share your learnings and insights with thousands of people online. To build a community of people that follow their curiosity and create a bottom-up, curated knowledge base. And if you want to monetise the work that you are putting in, you can do so on your own terms – without any ads that create adverse incentives and interrupt the experience.

We believe that the Curator Economy is a growing space that will be critical to how we engage with information and learning in the future. Interesting work is being done on multiple fronts, for example with community-curated networks of knowledge. So far efforts in this direction seem to focus on large, networked information structures. While these are clearly important, with Fount, we see the highest point of leverage at the top of the curation funnel: as a meta-layer filtering signal out of noise.

The Secret Masterplan

This is what we are planning to do – build something users love. This includes:

  • Build a way for users to fill the app with insights they love.
  • Enable users to connect insights in exciting ways.
  • Help users rediscover & explore their own and everybody else’s insights.
  • Turn users into curators.
  • Repeat.

Conclusion

Knowledge management almost always requires a lot of effort. Instead of playing with our knowledge driven by our curiosity, we built excessively complex systems that often feel like work to be maintained. We become stuck in silos and loose insights in the noise of social feeds. And instead of tapping into the existing curations of other users, our systems are just our own, private collections.

Our vision at Fount isbuilding a platform that allows you to benefit from the aggregated knowledge from curious people – experts, amateurs, autodidacts, polymaths and many more – from all around the world. A platform to save your insights from across the web in a simple and seamless manner, to (re-)discover your own insights and those of others, and, most importantly, to connect them and thereby create new and unique knowledge that then can be shared and built upon with others. It’s a place of serendipity, for you to follow your curiosity wherever it might lead you.

So let’s play.

Categories
Startup

Rob Fitzpatrick – The Mom Test

My Opinion

I’ve been positively surprised how well written and thought through this book is. In a very structured and precise approach Fitzpatrick describes how to have successful customer conversation.

Reading Recommendation: 8/10


The danger of the wrong conversation. Bad customer conversations aren’t just useless. Worse, they convince you that you’re on the right path.

Only facts matter. The measure of usefulness of an early customer conversation is whether it gives us concrete facts about our customers’ lives and world views. These facts, in turn, help us improve our business.

Don’t mention your business. If you just avoid mentioning your idea, you automatically start asking better questions. Doing this is the easiest (and biggest) improvement you can make to your customer conversations.

Rules of thumb: 

  • Opinions are worthless. Only facts matter. Only the market knows if something will work out.
  • Anything involving the future is an over-optimistic lie.
  • People will lie to you if they think it’s what you want to hear.
  • The more you’re talking, the worse you’re doing.

Mitigation: To avoid these biases, use the Mom Test: 

  • Talk about their life instead of your idea. 
  • Ask about specifics in the past instead of generics or opinions about the future. 
  • Talk less and listen more.

Dig deep on focus on the past and present. Ask how they currently solve X and how much it costs them to do so. And how much time it takes. Ask them to talk you through what happened the last time X came up. If they haven’t solved the problem, ask why not. Have they tried searching for solutions and found them wanting? Or do they not even care enough to have Googled for it?

Rule of thumb: People know what their problems are, but they don’t know how to solve those problems. That’s your job. 
The questions to ask are about your customers’ lives: their problems, cares, constraints, and goals. You humbly and honestly gather as much information about them as you can and then take your own visionary leap to a solution. Once you’ve taken the leap, you confirm that it’s correct (and refine it). It boils down to this: you aren’t allowed to tell them what their problem is, and in return, they aren’t allowed to tell you what to build. They own the problem, you own the solution. 

Rule of thumb: You’re shooting blind until you understand their goals.
Mitigation: “Why do you bother?” It’s great for getting from the perceived problem to the real one.
Example: Some founders I knew were talking to finance guys spending hours each day sending emails about their spreadsheets. The finance guys were asking for better messaging tools so they could save time. The “why do you bother” question led to “so we can be certain that we’re all working off the latest version.” Aha. The solution ended up being less like the requested messaging tool and more like Dropbox. A question like “why do you bother” points toward their motivations. It gives you the why.

Rule of thumb: Some problems don’t actually matter .
Mitigation: “What are the implications of that?” Good question. This distinguishes between “I-will-pay-to-solve-that problems” and that’s-kind-of-annoying-but-I-can-deal-with-it“ problems. Some problems have big, costly implications. Others exist but don’t actually matter. 

Rule of thumb: Watching someone do a task will show you where the problems and inefficiencies really are , not where the customer thinks they are .
Mitigation: “Talk me through the last time that happened.” Whenever possible, you want to be shown, not told, by your customers. Learn through their actions instead of their opinions. Being walked through their full workflow answers many questions in one fell swoop: 

  • How do they spend their days, 
  • What tools do they use, and who do they talk to? 
  • What are the constraints of their day and life? 
  • How does your product fit into that day? 
  • Which other tools, products , software, and tasks does your product need to integrate with?

Rule of thumb: If they haven’t looked for ways of solving it already, they’re not going to look for (or buy) yours.

Mitigation: “What else have you tried?” It’s easy to get someone emotional about a problem if you lead them there. 

Example: “Don’t you hate when your shoelaces come untied while you’re carrying groceries?” “Yeah, that’s the worst!” And then I go off and design my special never – come – untied laces without realising that if you actually cared, you would already be using a double – knot.

Rule of thumb: While it’s rare for someone to tell you precisely what they’ll pay you, they’ll often show you what it’s worth to them.

Mitigation: “How are you dealing with it now?” Good question. Beyond workflow information, this gives you a price anchor.

Two questions to end every conversation with: 

  • “Who else should I talk to?” 
  • “Is there anything else I should have asked?”

There are three types of bad data: 

  • Compliments 
  • Fluff (generics, hypotheticals, and the future) 
  • Ideas

Don’t listen to opinions. With the exception of industry experts who have built very similar businesses, opinions are worthless. You want facts and commitments, not compliments.

Ignoring compliments should be easy, but it’s not. We crave validation and ,as such, are often tricked into registering compliments as reliable data instead of vacuous fibs. 

Rule of thumb: Compliments are the fool’s gold of customer learning: shiny, distracting, and worthless.

Fluff comes in 3 cuddly shapes: 

  • Generic claims (“I usually”, “I always”, “I never”) 
  • Future – tense promises (“I would”, “I will”) 
  • Hypothetical maybes (“I might”, “I could”)

The worst type of fluff – inducing questions are:

  • “Would you ever?” (of course they might. Someday.) 
  • “Do you ever …”
  • “Would you ever …” 
  • “What do you usually …” 
  • “Do you think you …” 
  • “Might you …” 
  • “Could you see yourself …”

Be specific. While using generics, people describe themselves as who they want to be, not who they actually are. You need to get specific to bring out the edge cases.
Example: Let’s say you’re building a mobile loyalty app to help stores give deals and discounts to their most loyal customers and you hear the guy in line in front of you complaining: A bad conversation (pitching and accepting fluff): 
Them: “Which idiot decided it was a good idea to make me carry around a thousand cafe loyalty cards?” 
You: “Ohmygosh hi! I just so happen to be building a mobile app to help stores give out discounts to their most loyal customers so you’d never need to carry paper cards again. Do you think you would use something like that?” This is pretty much as bad of a question as you can find. You’ve revealed your ego and asked a “would you ever ” question. You’re begging for a false positive.

Startups are about focusing and executing on a single , scalable idea rather than jumping on every good one which crosses your desk .

Rule of thumb: Ideas and feature requests should be understood, but not obeyed.
Mitigation: When you hear a request, it’s your job to understand the motivations which led to it. You do that by digging around the question to find the root cause. Why do they bother doing it this way? Why do they want the feature? How are they currently coping without the feature? Dig.
Examples: 
“Why do you want that?” 
“What would that let you do?” 
“How are you coping without it?”
“Do you think we should push back the launch to add that feature, or is it something we could add later?” 
“How would that fit into your day?”
Questions to dig into emotional signals: 
“Tell me more about that.” 
“That seems to really bug you — I bet there’s a story here.” 
“ What makes it so awful?” 
“Why haven’t you been able to fix this already?” 
“You seem pretty excited about that — it’s a big deal?” 
“Why so happy?” 

The main source of compliment – creation is seeking approval, either intentionally or inadvertently. Doing it intentionally is fishing for compliments. In other words, you aren’t really looking for contradictory information. You’ve already made up your mind, but need someone’s blessing to take the leap. Symptoms of Fishing For Compliments: “I’m thinking of starting a business … so, do you think it will work?”

Rules of thumb: 

  • If you’ve mentioned your idea, people will try to protect your feelings.
  •  Anyone will say your idea is great if you’re annoying enough about it.

Mitigation: In short, remember that compliments are worthless and people’s approval doesn’t make your business better. Keep your idea and your ego out of the conversation until you’re ready to ask for commitments.

Rule of thumb: You should be terrified of at least one of the questions you’re asking in every conversation .

Rule of thumb: Start broad and don’t zoom in until you’ve found a strong signal, both with your whole business and with every conversation.

Two types of risks: 

  • Product risk – Can I build it? Can I grow it? 
  • Customer / market risk – Do they want it? Will they pay me? Are there lots of them?

Example: Video games are pure product risk. What sort of question could you ask to validate your game idea? “Do you like having fun? Would you like to have even more fun?” Practically 100 % of the risk is in the product and almost none is in the customer. You know people buy games. If yours is good and you can find a way to make them notice it, they’ll buy it. You don’t need to rediscover people’s desire to play video games.
Implications: What all this does mean is that if you’ve got heavy product risk (as opposed to pure market risk), then you’re not going to be able to prove as much of your business through conversations alone. The conversations give you a starting point, but you’ll have to start building the product earlier and with less certainty than if you had pure market risk.

Rule of thumb: You always need a list of your 3 big questions .

Separate your meetings. In Steve Blank’s original book on Customer Development he solves this by recommending 3 separate meetings: the first about the customer and their problem; the second about your solution; and the third to sell a product. In practice, however, this might be difficult. To even find a suitable number of users to talk to, Steve recommends starting with friendly first contacts. 

The power of casual conversations. Instead of scheduled meetings, casual conversations on meetups or conferences (or even some kind of private gathering for that matter) work just equally well. The conversations become so fast and lightweight that you can go to an industry meetup and leave with a dozen customer conversations under your belt, each of which provided as much value as a lengthy formal meeting .

Rule of thumb: Learning about a customer and their problems works better as a quick and casual chat than a long, formal meeting.

Rule of thumb: Give as little information as possible about your idea while still nudging the discussion in a useful direction.

Two concepts to differentiate between:
Commitment – They are showing they’re serious by giving up something they value such as time, reputation, or money.
Advancement – They are moving to the next step of your real – world funnel and getting closer to purchasing.

First customers are crazy. Crazy in a good way. They really, really want what you’re making. They want it so badly that they’re willing to be the crazy person who tries it first. Keep an eye out for the people who get emotional about what you’re doing. There is a significant difference between: “Yeah, that’s a problem” and “THAT IS THE WORST PART OF MY LIFE AND I WILL PAY YOU RIGHT NOW TO FIX IT.”

Steve Blank calls them earlyvangelists (early evangelists). In the consumer space, this is the fan who wants your product to succeed so badly that they’ll front you the money as a pre-order when all you’ve got is a duct – tape prototype. They’re the one who will tell all their friends to chip in as well. They’re the person reading your blog and searching for workarounds.

Keep your early customers close. Firstly, when someone isn’t too emotional about what you’re doing, they are unlikely to end up being one of your crazy first customers. Keep them on the list and try to make them happy, of course, but don’t count on them to write the first check. Secondly, whenever you see the deep emotion, do your utmost to keep that person close. They are the rare, precious fan who will get you through the hard times and give you your first sale.

Rules of thumb: 

  • If it’s not a formal meeting, you don’t need to make excuses about why you’re there or even mention that you’re starting a business. Just ask about their life.
  • If it’s a topic you both care about, find an excuse to talk about it. Your idea never needs to enter the equation and you’ll both enjoy the chat.

Testing your value proposition via landing pages. The value of these quantitative metrics might be doubtful. But they are certainly a great way to collect emails of qualified leads for you to reach out to and strike up a conversation with.

Generic launch. Paul Graham suggests that generic launch can be a solid start for the same reason. Get your product out there, see who seems to like it most, and then reach out to those types of users for deeper learning.

Organise meetups. For marginally more effort than attending an event, you can organise your own and benefit from being the centre of attention. Nobody ever follows this recommendation, but it’s the first thing I would do if I moved to a new industry or geography. It’s the fastest and most unfair trick I’ve seen for rapid customer learning. As a bonus, it also bootstraps your industry credibility.

When asking for user interviews: Vision / Framing / Weakness / Pedestal / Ask

  • You’re an entrepreneur trying to solve horrible problem X, usher in wonderful vision Y, or fix stagnant industry Z. Don’t mention your idea. 
  • Frame expectations by mentioning what stage you’re at and, if it’s true, that you don’t have anything to sell. 
  • Show weakness and give them a chance to help by mentioning the specific problem that you’re looking for answers on. This will also clarify that you’re not a time waster.
  • Put them on a pedestal by showing how much they, in particular, can help. 
  • Explicitly ask for help. 

Example Hey Pete, I’m trying to make desk & office rental less of a pain for new businesses (vision). We’re just starting out and don’t have anything to sell, but want to make sure we’re building something that actually helps (framing). I’ve only ever come at it from the tenant’s side and I’m having a hard time understanding how it all works from the landlord’s perspective (weakness). You’ve been renting out desks for a while and could really help me cut through the fog (pedestal). Do you have time in the next couple weeks to meet up for a chat? (ask)

Pay attention to your types of customers. If you’ve run more than 10 conversations and are still getting results that are all over the map, then it’s possible that your customer segment is too vague, which means you’re mashing together feedback from multiple different types of customers.

Rule of thumb: Keep having conversations until you stop hearing new stuff .

Focus! They say that startups don’t starve, they drown. You never have too few options, too few leads, or too few ideas; you have too many. You get overwhelmed. You do a little bit of everything. When it comes to getting above water and making faster progress, good customer segmentation is your best friend.

The danger of being too broad. If you start too generic, everything is watered down. Your marketing message is generic. You suffer feature creep. In their early days, Google helped PhD students find obscure bits of code. Paypal helped collectors buy and sell Pez dispensers and Beanie Babies more efficiently. Evernote helped moms save and share recipes.

Rule of thumb: If you aren’t finding consistent problems and goals, you don’t have a specific enough customer segment .

Mitigation: Before we can serve everyone, we have to serve someone. Otherwise, every debate over a new feature could be won by claiming , “Well , those guys would love it.” The reverse argument could be made to prevent any feature’s removal. Progress cannot be made. 

Example: Imagine that we’re building something for “students”. I’ve got a picture of an American undergraduate in my head, and maybe you picture a British grad student, but we manage to agree on features and start building.
We conduct 20 conversations with our customers. The feedback is inconsistent. Problem: We had conversations with 20 different types of customers. 

Start with a broad segment and ask: 

  • Within this group, which type of person would want it most? 
  • Would everyone within this group buy / use it, or only some? 
  • Why does that sub – set want it? (e.g . what is their specific problem) 
  • Does everyone in the group have that motivation or only some? 
  • What additional motivations are there? 
  • Which other types of people have these motivations?

Rule of thumb: Good customer segments are a who – where pair. If you don’t know where to go to find your customers, keep slicing your segment into smaller pieces until you do. If there isn’t a clear physical or digital location at which you can find your customer segment, then it’s probably still too broad.

Three criterias for your target customers. You’ll broaden your segment back out later. But your learning will go faster (and be more useful) for now by choosing someone who is specific and who also and meets the three big criteria of being reachable, profitable, and personally rewarding .

Don’t do it alone. A common anti – pattern is for the business guy to go to all the meetings and subsequently tell the rest of the team what they should do. Bad idea. Telling the rest of the team “What I learned” is functionally equivalent to telling them “What you’ll do.” Therefore, owning the customer conversations creates a de-facto dictator with “The customer said so” as the ultimate trump card.

When preparing, ensure you know your current list of 3 big questions. Figure them out with your team and make a point to face the scary questions.

Create a skeleton in advance. It’s easier to guide the conversation and stay on track if you have an existing set of beliefs that you’re updating. Spend up to an hour writing down your best guesses about what the person you’re about to talk to cares about and wants. You’ll probably be wrong, but it’s easier to keep the discussion on track and hit important points if you’ve created a skeleton. If you have an appropriately focused segment, then you’ll only rarely need to do this.

Rule of thumb: If you don’t know what you’re trying to learn , you shouldn’t bother having the conversation. All you’re really trying to figure out is: What do we want to learn from these guys?

How to share the results? On a logistical level, some teams have a quick chat about the results of each meeting as soon as they get back to the office. Others have longer weekly meetings to go through all the week’s notes and learnings.

How many participants for a user interview? Meetings go best when you’ve got two people at them. One person can focus on taking notes and the other can focus on talking.

How to take notes? 

  • When possible, write down exact quotes. Wrap them in quotation marks so you know it’s verbatim.
  • Use symbols for emotions 🙂 Excited 🙁 Angry 😐 Embarrassed
  • Use symbols for other key elements: ☇ Pain or problem (symbol is a lightning bolt ); ⨅ Goal or job-to-be-done (symbol is a soccer / football goal); ☐ Obstacle;  ⤴ Workaround; ^ Background or context (symbol is a distant mountain)
  • Use symbols for further important information: ⇪Feature request or purchasing criteria;$Money or budgets or purchasing process; ♀  Mentioned a specific person or company; ☆ Follow – up task

The process before a batch of conversations: 

  • If you haven’t yet, choose a focused, findable segment with your team 
  • Decide your big 3 learning goals 
  • If relevant, decide on ideal next steps and commitments 
  • Create a series of best guesses about what the person cares about 
  • If a question could be answered via desk research, do that first 

During the conversation: 

  • Frame the conversation 
  • Keep it casual 
  • Ask good questions which pass The Mom Test 
  • Deflect compliments, anchor fluff, and dig beneath signals 
  • Take good notes 
  • If relevant, press for commitment and next steps 

After a batch of conversations: 

  • With your team, review your notes and key customer quotes 
  • If relevant, transfer notes into permanent storage 
  • Update your beliefs and plans 
  • Decide on the next 3 big questions

Getting back on track (avoiding bad data): 

  • Deflect compliments 
  • Anchor fluff 
  • Dig beneath opinions, ideas, requests, and emotions