Forget data: The case for going with your gut instinct

Data analysis may be held up as the gold standard for making sound business decisions. But a recent study suggests gut instinct — when backed up by simple decision aids — may actually be better in situations of high uncertainty.

Examples of simple decision aids [referred to in the article as “heuristics”] could be tallying, or scoring. This is when everyone lists the good points of going with a certain option — be it a certain innovation, a new product line, making an underwriting decision, etc. — and the option with the most points is the winner.

Other simple decision aids to supplement gut instinct include, among other things, “experience” (for example, choosing the option the most experienced people in the team want) or “majority rules” (choosing the option most people want).

When combined with gut instinct, these simple aids may actually make for better decision-making than if managers rely on complex data analytics to help them chart their way through highly uncertain situations, say U.K. researchers Oguz A. Acar and Douglas West. The authors explain why in their blog, ‘When an Educated Guess Beats Data Analysis,’ published recently by Harvard Business Review.

“While the decisions based on data analysis brought about a good level of decision-making accuracy, the process was slow,” Acar and West say in their blog. “Managers who relied on their instincts together with some simple heuristics made decisions that were just as accurate but were undertaken much more quickly.

“That is, [decision aids] and gut feelings offered a better tradeoff in terms of decision-making speed and accuracy; the inclusion of analysis in the decision-making process did not bring about any meaningful improvement in accuracy while significantly reducing speed.”

The authors emphasize that they were analyzing decision-making under extremely uncertain conditions.

While their research focused mainly on creative industries, in the context of the property and casualty insurance industry, the study results could apply to decisions about underwriting risks, marketing new products, choosing between digital platforms, or setting strategic business priorities.

For their study, Acar and West collected data from 122 companies in creative industries (in advertising, digital, publishing, and software sectors) about their latest innovation projects. They chose these fields in particular due to high levels of uncertainty about customer reactions and an infinite variety of potential new products and product modifications.

They asked managers how they made their decisions, including the degree to which they relied on data, instinct, and simple decision aids. Then they asked the managers whether, in retrospect, they got the decision right, and how quickly it took them to arrive at the decision.

“The results first showed, to our surprise, despite the huge interest in big data, that the managers in our sample did not rely on analysis any more than on their instincts or some of the simple [decision aids]. The most commonly used heuristic, more than both analysis and instinct, was tallying.”

The authors cautioned that the success of instinct correlated with experience.

“One note of caution for managers who consider embarking on gut-based innovation decisions,” the authors state in their blog. “The effectiveness of their intuition might rely on prior experience. Prior research suggests that the effectiveness of intuition compared to analysis is contingent upon domain knowledge; experts in a domain are more likely to make better gut decisions.

“Managers with limited domain expertise might therefore be better off by refraining from extensive reliance on intuition. Our results suggest relying mainly on heuristic also presents a viable alternative.”

Feature image courtesy of iStock.com/RichVintage