Good decision-making starts with one simple habit: Commit to your default decision up front. You’ll choose it by making a judgment call about which action is the lesser evil under ignorance. Choosing a default action is difficult for decision-makers who aren’t used to it. You ask yourself, “If I see no additional data beyond what I’ve already seen, what will I do?” Answering this takes strength of character — you can’t punt it to the data. You really have to think about the business problem and answer truthfully, “What am I going to pick if I have to make the decision right now?”
As a statistician, I appreciate the quote by applied statistics pioneer W. Edwards Deming, “In God we trust. All others bring data.” But as a social scientist, I’m compelled to warn you that many decision-makers chase data with too much zeal, running from ignorance but never improving their decisions. Is there a way to land in the sweet spot? There is, and it starts with one simple decision-making habit: Commit to your default decision up front.
The key to decision-making is framing the decision context before you seek data — a skill that unfortunately is not usually covered in data science courses. To learn it, you’ll need to look to the social and managerial sciences. It’s unfortunate that we don’t teach it enough where it is most needed: as a skill for leading and managing data science projects. Even in statistics, which is the discipline of making decisions under uncertainty, most of the exercises that students encounter already have the context pre-framed. Your professor usually creates the hypotheses for you and/or frames the question so there’s only one right answer. Wherever there’s a right answer, the decision-maker has already blazed that trail.
Many decision-makers think they’re being data-driven when they look at a number, form an opinion, and execute their decision. Unfortunately, such a decision will be “data-inspired” at best. Data-inspired decision-making is where we swim around in some numbers, eventually reach an emotional tipping point, and then decide. There were numbers near that decision somewhere, but those numbers didn’t drive the decision. The decision came from somewhere else entirely. It was there all along in the unconscious biases of the decision-maker.
If you’re undisciplined in your attempts to use data for decision-making, your approach is susceptive to cognitive bias.
One major problem with data-inspired decision-making is confirmation bias, which influences how the decision-maker will perceive facts in light of what they already believe. If you’re free to move the goalposts after you find out where the data landed, then that’s exactly what you’ll do, unconsciously. The solution is to set the goalposts in advance and resist the temptation to move them later.
That’s why behavioral economists are trained to set decision criteria in advance of information. Since it’s the best antidote to confirmation bias, many of us have it ingrained as a habit. We can’t help but ask ourselves, for example, the maximum amount that we’d pay for a ticket before we look up the price.
By leaving the decision criteria open, you’re free to interact with the data selectively to confirm the choice you’ve already made in your heart of hearts. You’re merely using the data to feel better about doing what you want to do anyway. Most humans do this without even realizing it.
Another human foible to avoid is the Ikea effect. You’re experiencing this effect if you overvalue something as a result of having put effort into it. Simply put, when people invest time into a project, they’re likely to fall in love with it, even if what they’ve built is a pile of poisonous rubbish, and this will change how they perceive it. They’ll start bargaining with themselves, “Oh, but the performance of my new prototype is not so bad, I could still release this thing…” And that’s how horrible things get foisted upon the world.
To avoid falling victim to these effects — to be truly data-driven — order matters! You need to frame the decision context up front. And the first part of that process is determining what you’re planning to do in the absence of further data. (Will you buy/launch/medicate/proceed or not buy/launch/medicate/proceed by default?). That’s called a default action and you’ll choose it by making a judgment call about which action is the lesser evil under ignorance.
Choosing a default action is difficult for decision-makers who aren’t used to it. You ask yourself, “If I see no additional data beyond what I’ve already seen, what will I do?” Answering this takes strength of character — you can’t punt it to the data. You really have to think about the business problem and answer truthfully, “What am I going to pick if I have to make the decision right now?”
For example, “Here’s a new medicine or new machine learning system. I don’t know if it works. By default, shall I launch it or shall I not?” (Most people would probably say not, except maybe those who feel that a machine learning system looks great on a resume.)
Once you’ve specified your default action, you’re cleared to start thinking about data. But even then, the first step isn’t to go collect or analyze it. After deciding what your default action will be in the absence of new information, you’re going to think about how you would react to the data when it arrives. What form should it take to talk you out of your default action? To answer this question, you need the ability to imagine various states of the world, identify whether the default is the right action for each of them, and then create a metric that can tell you which of these worlds you inhabit. Finally, you’ll consider what magnitude of evidence is required to sway you from the default and what your tolerance for risk is. Only then is it time to get into the numbers.