What Is Bias in Machine Learning?

Bias in machine learning occurs when AI models and algorithms include systematic errors that lead to unfair outcomes, such as favoring one arbitrary group over others. You may be able to trace AI bias back to the data, the way developers write algorithms, and other sources of human error. 

All people, yourself included, have biases. Forming a bias helps you recognize patterns and make decisions about how to move through life. In fact, bias helps you make quick decisions. You can imagine when bias might be helpful. For example, your prehistoric ancestors may have witnessed a member of their community get sick after eating a certain food and consequently develop a bias against that food to protect themselves from a possible poison. 

But in today’s modern world, many of the biases that society develops are harmful and not helpful. This is particularly true when a person holds unconscious biases about marginalized groups of people, which leads to discrimination. Ideally, artificial intelligence would be free from these unconscious biases, relegating such errors as a relic of a biological brain. 

Unfortunately, AI models and algorithms can perpetuate human bias in many different ways, such as by taking as fact biases present in training data or in human error interacting with the model, for example. This reduces the accuracy of AI and can have real-world consequences for companies, individuals, and society at large. Discover examples of bias in machine learning, where AI bias comes from, and what you can do to counteract machine learning bias. 

What is bias in machine learning?

Bias in machine learning happens when AI algorithms develop bias, for example, against marginalized individuals like people of color and women. This tends to occur because researchers train AI models with available data that often reflects the historical biases of the time or the biases of the researchers. In other cases, biases are continued through human error when interacting with AI analysis or data. Yet, in other cases, biases exist because of the way that humans frame the problem they want to solve. For example, you might want to create an AI program that helps you locate the best restaurant in a city. You first have to decide what constitutes “the best,” which means the results may skew towards the foods you like the best instead of catering to a more diverse perspective. 

AI bias can discriminate against individuals, and it can give businesses and organizations inaccurate results. For example, imagine an AI model designed to pick the most creditworthy applicants for an apartment building. If the model demonstrated bias against non-white applicants, it would harm both the applicants facing discrimination and the company, which is unable to accurately determine which applicants would be the most creditworthy. Inaccurate artificial intelligence could lead to making poor decisions, offering a bad experience to your customers, or outright discriminating against groups of people. 

Example of AI bias

AI bias finds its way into systems if researchers and professionals do not take steps to watch for and correct bias. One example of bias in machine learning with real-life implications is a tool that Amazon released in 2014 and pulled in 2015. 

This tool was an AI application that could analyze the resumes submitted to a hiring manager and return the top five most qualified applicants. Amazon’s team trained the tool using data from 10 years of hiring history within the company. In analyzing this data, the AI tool trained itself that men were preferable candidates over women, partially due to the over-representation of men’s applications in the training data. When the program encountered a resume with words that indicated it was a woman’s application, such as being a member of a woman’s organization, the AI penalized that application and ranked it lower than a resume without those terms. Amazon edited the program to avoid bias but ultimately ended the project. 

Machine learning bias could also lead to: 

• Inaccurate medical diagnosis: When medical professionals use AI to diagnose disease or illness, the results could be inaccurate due to underrepresenting certain populations in the training or research data.

• Biases in search results: AI bias could lead search engine algorithms to make decisions about what results to show you based on demographics like your race or gender. 

• Racial profiling in policing: When police and other safety officers use AI to determine what areas of their community require more police presence, they may rely on historical data that demonstrate patterns of racial profiling, thereby perpetrating that bias. 

How does bias in machine learning happen?

AI models can develop bias for a number of reasons such as flawed training data, anachronistic historical data, or the model’s own confirmation bias. Consider these types and sources of bias in machine learning: 

Training data bias: When bias exists in training data, it can appear in the AI model. 

Historical bias: If you use historical data to train your AI, it could perpetuate bias that is common in that era. 

Algorithmic bias: Algorithmic bias refers to bias resulting from faulty training data. It can also occur if a programmer indoctrinates the algorithm’s decision-making ability with their own conscious or unconscious bias. 

Cognitive bias: Cognitive bias is an error based on the point of view of the human programming an AI, such as favoring US citizens over a global perspective. 

Confirmation bias: When an AI uses its own bias or faulty reasoning to confirm a bias as fact, it is called a confirmation bias. 

Exclusion bias: In some cases, exclusion bias appears in machine learning after researchers exclude data they don’t believe is relevant but would, in fact, change the AI’s conclusion. 

Sample bias: When researchers don’t include enough training data for their AI or if the training data they include doesn’t represent the population as a whole, it creates a sample bias. 

Who works against bias in machine learning?

Different types of professionals work in the field of machine learning and artificial intelligence, attempting to develop better algorithms. If you’d like to consider a career working to prevent bias in machine learning, three potential careers for you to explore include AI researcher, data scientist, and machine learning engineer. 

AI research scientist

Average annual salary in the US (Payscale): $143,184 [1]

Job outlook (projected growth from 2023 to 2033): 26 percent [2]

As an AI research scientist, you will use artificial intelligence to research and find solutions to real-world problems. You will determine the best strategies to create, develop, and train algorithms, as well as apply AI analysis to complex problems. 

Data scientist

Average annual salary in the US (Payscale): $101,107 [3]

Job outlook (projected growth from 2023 to 2033): 36 percent [4]

As a data scientist, you will use data to make recommendations to businesses and organizations. In this role, you may work with or create AI algorithms to interact with data, in addition to gathering raw data, preparing data for analysis, and presenting your findings. 

Machine learning engineer

Average annual salary in the US (Payscale): $119,364 [5]

Job outlook (projected growth from 2023 to 2033): 26 percent [2]

As a machine learning engineer, you will create, test, and troubleshoot machine learning applications for clients or to solve problems. In this role, you may also identify the proper data sets to utilize when training your algorithms.

How to address bias in machine learning

Just like discrimination in society, addressing bias in machine learning is a process that researchers and AI developers must continue to pursue. Important conversations are ongoing to determine the best path forward, such as defining what fairness will mean in artificial intelligence applications. 

Efforts to address bias in machine learning can be roughly sorted into measures you can take prior to running the algorithm, such as making sure your data is as unbiased as it can be, and measures you can take after processing the data, such as rebalancing the data against a standard of fairness. A third category also exists where you can add fairness constraints to the algorithm itself.

Other strategies for addressing bias in machine learning include: 

• Improving your data

• Using data from diverse sources or representing a diverse population

• Testing and analyzing algorithms for biases proactively

• Designing more complex models

• Developing a thorough understanding of “fairness” in algorithm development 

• Establishing AI policies that combat bias

• Keeping humans in the review process to look for errors

Learn more about machine learning on Coursera

Bias in machine learning is a problem that can impact individuals, companies, and organizations. If you want to learn more about how to fight bias in machine learning, consider an online course. Explore Unpacking Unconscious Bias in the Workplace, the IBM Machine Learning Professional Certificate, or the Machine Learning Specialization offered in partnership between DeepLearning.AI and Stanford.