4 Cognitive Biases Every Product Manager Needs to Combat
Product managers have a difficult job. They live at the intersection of vastly different teams, caught in between the business side and the development side of the company. They are responsible for carrying the burden — feature requests — of everyone from customer service reps and sales managers to the UX team and data scientists. With so many sources of information, PMs need to be especially mindful of the cognitive biases that can plague their thinking.
With an awareness of how our brains tend to trick and trap us, we can find a path to clearer thinking and better decision-making. Here are four specific cognitive biases you should be mindful of.
Survivorship bias is the over-emphasizing of prior successes — whatever “survived” and made it through a process. Survivorship bias narrows your field of vision, so that you’re making decisions based on incomplete information, ignoring key evidence from the failures and cases that don’t make it through.
For instance, new parents are bombarded with plenty of do’s and don’ts, suggesting that peril is around every corner for their little ones. “Put your baby on his back or he might die of SIDS.” “Mount that shelf to the wall, lest it topple over on her.” While nervous parents may be ready to heed such advice, others may chime in. “When I was growing up, we didn’t worry about any of that, and we turned out just fine.” Yep. Any person for whom that was not true (i.e., crushed by that dresser) is not around to tell the tale. Thus, the elders who remain to warn new parents are survivors of prior mistakes. They have access to incomplete information that can lead to incorrect conclusions about safety.
Your existing users are survivors too.
When you go to learn about how your product is working from them, you always have to weigh the fact that they made it through. Be careful of listening too closely to your current customers. Although they may provide valuable insights you can leverage for engagement, that insight is limited to that type of customer at that stage in the process. Be wary of trusting them for onboarding advice, which may be better gathered from new users who may ultimately churn. Remember to ask yourself who has self-selected into the group you are evaluating, and who you are overlooking.
As humans, we often make inferences about the world and use those inferences to be efficient in future, novel scenarios. This makes a lot of sense. After all, what good would it do us if we had to evaluate every new piece of information without any regard for prior experience? But there’s a dark side to this process. Confirmation bias is the human tendency to interpret new information as a confirmation of our existing beliefs and ignore it if it challenges our existing beliefs. “Obama wasn’t born here” or “Trump colluded with Russia” are all too easy to believe when you enter with that belief.
Product managers, especially those who strive to be data-driven, are at risk of making suboptimal decisions as a result of confirmation bias.
Dashboards make this all too easy, when PMs are able to query the data on a whim without the rigor of establishing concrete hypotheses. Do you suspect that the new feature you just implemented will increase user engagement? A quick glance across the dashboard, and your eye might be caught by all of the positive metrics. Time in app is up 3%! Lots more taps! First, it’s not clear whether a 3% increase is actually statistically significant. Here, you might be overemphasizing the value of the information which agrees with your suspicion. Be rigorous! Run the t-test to confirm. Second of all, your eyes may be darting past all of the metrics that haven’t changed much since the last build. Here, you may be ignoring other important information you would need to accurately evaluate the data and make informed product decisions. For this reason, product managers should set their key measures of performance ahead of time and examine them coherently.
The Bizarreness Effect
The Bizarreness Effect occurs when bizarre or out of the ordinary material is remembered better than what is ordinary or common. Our brains tend to boost the importance of things that are unusual or surprising. Alternatively, we tend to skip over information that we think is ordinary or expected. It’s easy to see how this (and the closely related Von Restorff effect) are leveraged by marketers. Imagine that you’re watching Sunday football with announcers yelling and the commercials blaring. Suddenly, you notice silence. A clever advertiser features a commercial with subtitles, and no sound, which quickly stands out next to the cheers and jingles of other commercials.
Product managers who are paying attention to user behavior are especially likely to run into the bizarreness effect. It tends to come in the form of giving additional attention to unusual user behavior. Many users of Uber use the service for short drives, trips home from the bar, or even their daily commute. But the user that takes an Uber from Seattle to Phoenix? That will catch your attention. The parent who says “Meh, I don’t care if my babysitter is background-checked as long as I get a good vibe;” the person who only lives in Airbnbs, in lieu of having a home or apartment; the new member to the coworking space who only joined to fill up 5 thermoses of coffee each day, these unusual scenarios will catch your attention more than others and they will be remembered more vividly.
This can affect your product management strategy in a number of different ways. First, and most obviously, you may tend to think this problem is more common than it is or devote more mental resources to understand it than is merited.
While the behavior is uncommon, this may lead you to over prioritize it.
Do you need to put up a sign saying that coffee is limited to 2 servings per day? Probably not, because it’s probably unusual. Second, you may also remember it more vividly, putting you in the position to pay extra (undue) attention to similar cases in the future. The second time you year of someone living only in vacation rentals, you might declare “Ah ha! It’s a trend!” Put another way, this sets you up for confirmation bias, causing you to seek similar cases in the future and overemphasize those as well. When something out of the ordinary strikes, remember to ask yourself whether this is common enough to deserve special attention.
The Sunk Cost Fallacy
The sunk cost fallacy occurs when you continue a behavior simply because you’ve already invested time, energy, or money into it, even when it’s not optimal to do so. (Really, it’s the consequence of a host of other cognitive biases, not a cognitive bias itself.) Have a beloved shirt that you have to sew a button back onto? A year later, when the shirt has lost your affection, you might find it a bit harder to part with. “Well, if I took the time to sew that button back on, I must really love it.” Back into the closet it goes, without being worn. Started planning a talk for your company, only to realize that the talk would have made more sense 6 months earlier? Two hours into prepping your talk, it’s hard to come to grips with that realization. So, you give the talk anyway, even though it requires another 5 hours of preparation. Our commitment to our past behavior makes it hard to be logical about how to invest our future behaviors.
A key element of good decision making is knowing when to quit.
Product managers are victims of this as well. Imagine you had a product built in PHP and it once worked flawlessly. Your team spent effort recruiting coders with relevant experience and built internal documentation to facilitate onboarding of new developers. But now it’s time to scale and PHP may be too slow. The investment you’ve put into PHP is substantial, making it harder to be logical about a potential switch to a more nimble system like Node.js. For this reason, technical product managers might (unwisely) resist the urge to switch to a more future-proof solution, and instead spend additional resources on implementing custom solutions to make the system last a little longer. After those investments are in place, a switch will be even less appealing: “We just spent a whole quarter making this system work. We can’t switch NOW!” The sunk costs fallacy might rear its ugly head when you continue iterating on a feature that isn’t sticking or when you keep a newly implemented design that didn’t improve your KPIs as expected. A key element of good decision making is knowing when to quit. Ask yourself: “If our code was lost and we had to build this again from scratch, is this the feature set or system I would build?”
How to Reduce Errors in Judgment
Despite our best efforts, humans will always commit these logical errors. They are the results of systems that generally work. Sometimes though, they cause us to make inferior decisions. Product managers who want to overcome them should consider educating themselves here and through general programs like the Center for Applied Rationality. Bringing in external support can help as well. User research agencies like PhD Insights can help offer a less partial perspective on product decisions, especially as it relates to sunk costs. By becoming aware of these biases, you are in a better position to seek complete, accurate information and make the best possible product decisions.