Machine Learning: The Cure for Decision Bias


We all know that machine learning offers unprecedented efficiency and accuracy in analyzing data to make predictions, recommend actions, and automate simple tasks. But, machine learning delivers an additional benefit that, while less widely discussed, is every bit as vital: objectivity.

No matter how vigorously we strive to remain objective, every human decision is subject to cognitive biases, limitations on the thinking process that lead to flawed judgments. With machine learning, organizations can remove — or at least lessen the impact of — the bias factor in business intelligence, ensuring that all information is accurately considered and weighted to produce an optimal outcome, every single time.

The Burden of Bias

In an ideal business world, workers would approach each decision by examining all the available information and drawing the most logical conclusion. While this approach has served us well for hundreds of years, it presents two key problems.

First, the human mind can only hold so much information at once, raising the possibility of some crucial data being left out of the decision-making process. Second, every one of us brings to every decision our own distinct sets of biases which, despite our best efforts to be purely rational, inevitably affect the outcome.

Over the years psychologists have identified hundreds of human biases, and here are just a few that affect decisions across your organization:

Anchoring bias: We tend to rely more heavily on the first piece of information we encounter.

The “bandwagon effect”: We tend to rely more heavily on what others in our circles are doing or have done in the past.

The “ostrich effect”: We tend to filter out negative information and uncomfortable facts (“bury our heads in the sand”).

Human biases affect thousands of business decisions every day, from the CEO deciding whether to scrap a product line to the mail room worker deciding what to do with an illegible envelope. And many of those decisions have a direct impact on your bottom line.

The Machine Learning Solution

While technology will never replace humans completely, machine learning enables us to leverage capabilities that humans simply cannot match, serving the BI process in two key capacities:

Comprehensive analysis and accurate pattern recognition: A truly optimized decision takes into account all the information available — a task that, depending on the subject matter, could be impossible for any human to accomplish. Also, analyzing data encompasses a quest for patterns, and cognitive biases could lead a human to overlook certain patterns or to look for connections that might not be completely valid. Not only are machines capable of analyzing terabytes of data in minutes or even seconds, but they can identify patterns that even the most capable humans might miss.

Bias-free outcomes: Unlike humans, machines bring zero “baggage” to the task of predicting future developments or recommending courses of action. Each person sees and understands data differently, but the machine knows only one approach: objective analysis of all information according to a fixed set of parameters. With today’s advanced analytics systems, those outcomes can be translated into valuable insights, delivered to everyday users within their daily workflows.

The Next Frontier of BI?

As business intelligence tools have evolved to make data more accessible and more understandable, data-driven decision-making has made its way into the hands of users across the organization, regardless of technical or analytical background. And while “data democratization” has empowered decision makers at all levels with a powerful tool, those decisions are still subject to the users’ cognitive biases. By bringing machine learning into the mix, organizations can reduce the impact of biases, enabling the organization to achieve a new level of business intelligence — one that uses data to the full extent of its potential.

By Arvin Hsu, (above) Senior Director of Data Science and Machine Learning for GoodData.

This content is original to AI Trends.