Understanding ML Fairness: Causes of Bias & Strategies for Achieving Fairness

The innovation potential seems boundless in a world driven by powerful advances in machine learning (ML). ML has transformed the way we live and interact with technology, from self-driving cars to personalized recommendations and assistants. However, behind these breakthroughs lies a pressing concern: fairness in ML. Imagine a scenario where a resume screening system trained on historical data reflects discriminatory hiring practices. Suppose the historical data disproportionately favor applicants from specific gender, race, or educational backgrounds. In that case, the ML model may learn to prioritize those attributes, resulting in discriminatory outcomes. Qualified candidates from underrepresented groups may be unfairly filtered out or receive lower rankings, perpetuating existing inequalities in the job market. These real-life instances underscore the critical importance of addressing bias and achieving fairness in ML systems.

In this article, we will delve into the causes of bias in ML models and explore effective strategies for achieving fairness. By understanding ML fairness’s ethical, legal, and regulatory dimensions, we can build transparent, accountable, and unbiased AI systems.  Let’s explore the intricacies of ML fairness and equip you with actionable insights and best practices to incorporate in the development and deployment of ML models.

Source: Pixabay

In machine learning, bias refers to systematic errors in a model’s predictions. Several factors contribute to bias, including the data used for training, the design of the model, and its usage. ML bias occurs when ML models consistently favor or discriminate against specific individuals or groups based on attributes such as race, gender, age, or other protected characteristics. Recognizing and addressing bias is essential to uphold fairness and equity in ML systems.

There are many sources of bias in machine learning models. Some of the most common sources include:

• Data collection and sampling: Bias can infiltrate a model if the training data does not represent the real-world population. For instance, if a loan application dataset primarily consists of applications from a particular demographic, the model may exhibit bias against applicants from underrepresented groups.
• Feature selection and engineering: Careless or biased selection of features during model development can introduce bias. For instance, including race in a job application model can lead to discriminatory outcomes favoring certain races.
• Algorithmic biases: Some machine learning algorithms inherently possess biases due to their design. Decision trees, for example, can be more prone to bias than random forests, potentially resulting in unequal treatment.
• Labeling biases: Biases can arise from inaccurate or subjective labeling during the training data annotation process. If medical diagnoses are predominantly labeled for one gender, the model may exhibit bias against the other gender.
• Human biases in the development process: Biases can be inadvertently introduced during various stages of model development, such as problem formulation, data preprocessing, model selection, and evaluation. The development team’s diverse perspectives and awareness of biases play a crucial role in mitigating such biases.

Understanding these sources of bias in ML models is crucial for building fair and unbiased systems. By addressing these causes, we can work towards creating ML models that promote fairness and inclusivity.

Biased machine learning (ML) models can have significant negative consequences that impact individuals, organizations, and society. Let’s explore some of these consequences:

1. Discrimination and unfair treatment: Biased ML models can perpetuate discriminatory outcomes, resulting in individuals or groups being mistreated or disadvantaged. For example, a biased loan model may systematically deny loans to qualified individuals of specific racial or ethnic backgrounds.
2. Legal and regulatory implications: Biased ML models can violate laws and regulations prohibiting discrimination based on race, gender, or disability. Organizations using biased models may face legal consequences and regulatory scrutiny, leading to financial penalties and reputational damage.
3. Ethical considerations: The presence of bias in ML models raises ethical concerns, including the infringement of privacy and the violation of individuals’ rights to fair treatment. It is essential to address discrimination to ensure equitable and responsible use of ML technologies.
4. Damage to public trust and company reputation: Biased ML models can erode public confidence in technology companies and AI systems. When the public becomes aware of biased outcomes or discriminatory practices, it can diminish trust, negative sentiment, and lost revenue for the organizations involved. Maintaining public confidence in the fairness and reliability of ML models is crucial.

By understanding the potential consequences of biased ML models, we can recognize the importance of striving for fairness, transparency, and accountability in developing and deploying ML systems. The next sections will explore strategies and best practices to mitigate bias and promote fairness in ML models.

Fairness in machine learning (ML) encompasses different definitions that capture various aspects of equitable outcomes. Let’s explore some of the most common definitions:

• Demographic parity:  It focuses on ensuring the model’s predictions are independent of the protected attributes of the individual, such as race, gender, or age.  Hence,  achieving equal representation of different demographic groups in the outcomes of an ML model. For example, if a model predicts loan default, demographic parity would require the same probability of a positive prediction for all races.
• Equal opportunity: Equal opportunity aims to eliminate bias regarding false negatives, ensuring that ML models give individuals from different groups equal chances or opportunities. It ensures that the model’s predictions have the same positive predictive value for all groups. In other words, the model should be equally likely to predict a positive outcome for people of all races, genders, or ages, even if they have different rates of the outcome being predicted.
• Equality of odds is a fairness metric that addresses the disparities in false positives and false negatives. In other words, it ensures that the probability of a qualified applicant being hired is the same regardless of their protected attributes, such as race, gender, or age. Similarly, the likelihood of an unqualified applicant not being employed should be the same regardless of their protected attributes.
• Calibration within groups: This focuses on calibrating the ML model’s predictions within specific subgroups defined by protected attributes. It ensures that the accuracy or confidence of the model’s predictions is balanced across different groups, preventing over- or underestimation of outcomes for any particular subgroup

When assessing fairness in machine learning (ML) models, a variety of metrics can be used to measure the presence or absence of biases quantitatively. Here are some of the most commonly used fairness metrics:

• Confusion matrix-based metrics:
  Confusion matrices are a way of visualizing the performance of a machine learning model. They show the number of true positives, false positives, true negatives, and false negatives. Some confusion matrix-based metrics for measuring fairness include:
  Accuracy: Accuracy is the fraction of correct predictions. However, it is not a good measure of fairness because it does not consider the distribution of positive and negative labels.
  Precision: Precision is the fraction of positive predictions that are actually positive. It is a good measure of fairness because it measures how often the model correctly identifies positive cases.
  Recall: Recall is the fraction of positive cases that are correctly identified. It is a good measure of fairness because it measures how often the model identifies all positive cases.
• Disparate impact ratio:
  The disparate impact ratio measures how much the model’s predictions differ between groups. It is calculated by dividing the false positive rate for one group by the false positive rate for another group. A disparate impact ratio of 1 indicates that the model’s predictions are the same for both groups. A disparate impact ratio greater than 1 indicates that the model is more likely to predict a positive outcome for one group than another. A value close to 1 indicates fairness, while significantly higher or lower values may indicate bias. This metric is beneficial for evaluating employment and lending decisions.
• Equality of odds ratio: The equality of odds ratio measures how much the model’s predictions differ between groups, considering the true positive rate. It is calculated by dividing the false positive rate for one group by the false positive rate for another group and then dividing by the difference in true positive rates between the two groups. A value of 1 indicates that the model’s predictions are fair. A value of less than 1 indicates that the model is more likely to predict a positive outcome for one group than another. A ratio close to 1 signifies fairness, while significant deviations indicate potential bias in the model’s predictions.
• Calibration plot: A calibration plot evaluates the calibration of the model’s predicted probabilities against the observed probabilities. It helps identify if the model is well-calibrated across different groups. Deviations from the ideal diagonal line suggest bias in the model’s confidence or accuracy.

It is important to note that no single metric is best for measuring fairness. Different stakeholders may have other priorities, and it may be necessary to consider multiple metrics when assessing an ML model.

Testing. CI/CD. Monitoring.

Because ML systems are more fragile than you think. All based on our open-source core.

Install Open SourceBook a Demo

Visualizing fairness in machine learning (ML) models is essential for understanding and assessing biases. Here are some practical ways to visualize fairness:

• SHAP (Shapley Additive Explanations) plots:  are valuable for visualizing fairness in ML models. They show how individual features contribute to predictions, aiding fairness assessment. SHAP plots can be used in the following ways:
• Individual-Level: They explain predictions for specific instances, uncovering disparities in feature contributions across demographic groups.
• Group-Level: Aggregating SHAP values for a group reveals collective feature influences, identifying if specific groups are disproportionately affected.
• Summary: Displaying average SHAP values across the dataset highlights features with stronger effects on predictions for different demographic groups.
• Fairness-Aware: Customizing SHAP plots to emphasize fairness-related attributes exposes bias in the model’s treatment of attributes.
  

  SHAP waterfall Plot, Source Aboze Brain John

SHAP plots provide interpretable insights into fairness by examining feature contributions and uncovering potential biases. By utilizing these plots, we can better understand fairness in ML models and take informed actions during development and deployment. Read more: A Comprehensive Guide into SHAP (SHapley Additive exPlanations) Values

• Calibration curves:
  Calibration plots visualize the calibration of a model’s predicted probabilities compared to the observed probabilities. These curves plot the average predicted probabilities against the actual outcomes, often in the form of a line graph. Deviations from the ideal diagonal line suggest bias in the model’s confidence or accuracy. It helps identify if the model is well-calibrated across different groupsPractitioners can assess the model’s calibration across different groups by comparing the curve to the ideal diagonal line. Calibration curves are especially useful for determining if the model’s predicted probabilities are well-calibrated and unbiased across various subgroups.
  

  Calibration Curves, source: Mattia Cinelli

• Intersectionality plot: Intersectionality is crucial in comprehending the intricate nature of fairness and bias within machine learning models. It acknowledges that individuals possess multiple social identities, such as race, gender, and age, which intersect and interact, leading to distinct experiences and potential biases. Intersectionality plots visually represent how these intersecting characteristics can influence outcomes. Here’s an example of how an intersectionality plot can illustrate the impact of race and disability on income probability
  

  Intersectionality plot, source: Angelia Wang et al.

  An intersectionality plot showcases the interplay between race and disability and its effect on the probability of income. By examining this plot, we can discern the combined influence of these two protected characteristics on the likelihood of earning income. The plot provides a visual understanding of how race and disability interact, contributing to the disparities observed in income outcomes. Through intersectionality plots, we gain valuable insights into the simultaneous effects of multiple dimensions of identity and their implications for fairness. By acknowledging and visualizing the complex interrelationships between protected characteristics, we can address and rectify biases arising from these intersections, thereby fostering more equitable outcomes within ML models.

It’s important to note that these visualizations alone cannot definitively determine the fairness of an ML model. It’s crucial to employ multiple visualizations and consider results from various metrics to assess fairness comprehensively. These visualization techniques provide valuable insights into the presence and nature of biases, empowering stakeholders to identify areas for improvement and take appropriate actions to promote fairness and mitigate biases in ML models.

Achieving fairness in ML models requires implementing various strategies such as:

Bias mitigation techniques

• Pre-processing: Pre-processing techniques can be used to reduce bias in the data before it is used to train the ML model. Some common pre-processing techniques include:
  Data augmentation: Data augmentation involves creating new data points by artificially varying the existing data points. This can help to reduce bias by making the data more representative of the real world.
  Re-sampling: Re-sampling involves oversampling or undersampling data points from different groups to balance the distribution of data points. This helps reduce bias by ensuring that all groups are represented equally in the data.
  Feature selection and transformation: Feature selection involves selecting only the most essential features for the ML model. Feature transformation involves transforming the features to make them more suitable for the ML model. This can reduce bias by removing features that are not relevant to the prediction task or that are correlated with protected attributes.
• In-processing: In-processing techniques can be used to reduce bias during the training of the ML model. Some common in-processing techniques include:
  Algorithmic modifications: Algorithmic modifications involve modifying the ML algorithm to make it less sensitive to bias. This can be done by using algorithms that are designed to be fair or by adding bias-reducing constraints to the algorithm.
  Fairness-aware optimization: Fairness-aware optimization involves optimizing the ML model for fairness while maintaining accuracy. This can be done using fairness-aware loss functions or fairness-aware regularization techniques.

• Post-processing: Post-processing techniques can be used to reduce bias after the ML model has been trained. Some common post-processing techniques include:
  Threshold adjustment: Threshold adjustment involves adjusting the threshold for classification predictions to reduce bias. This can be done by adjusting the threshold so that the same proportion of people from different groups are classified as positive.
  Cost-sensitive classification: Cost-sensitive classification involves assigning different costs to different types of misclassification errors. This can be used to reduce bias by making the ML model more sensitive to misclassification errors for certain groups.
  Fairness-aware ensemble methods: Fairness-aware ensemble methods involve combining the predictions of multiple ML models to reduce bias. This can be done by using ensemble methods that are designed to be fair or by using fairness-aware aggregation techniques.

• Continuous monitoring and evaluation: It is important to continuously monitor and evaluate the fairness of ML models throughout their lifecycle. This can be done by using fairness metrics to measure the fairness of the model’s predictions.
• Iterative model improvement: Improving the fairness of ML models iteratively is essential. This can be done by using the results of fairness metrics to identify and address bias in the model.
• Model governance and documentation: It is important to have a process for governing and documenting the fairness of ML models. This process should include steps for collecting and managing fairness metrics, tracking changes to the model, and communicating the model’s fairness to stakeholders.

• Importance of interdisciplinary collaboration: Collaborating with domain experts and diverse teams is important when building ML models. Domain experts can help to ensure that the model is designed to meet the needs of the real world, and diverse teams can help to identify and address bias in the model.
• Benefits of diverse perspectives: Diverse perspectives can help to identify and address bias in ML models. Different people have different experiences and perspectives, which can lead to different insights about the potential for bias in a model.
• Ensuring stakeholder input: It is vital to ensure stakeholders have input into the design and development of ML models. This helps ensure that the model meets the needs of the stakeholders and that it is fair to all users.

By following these strategies, it is possible to build ML models that are fair and equitable for all users.

Legal Frameworks for ML Fairness

• Anti-discrimination laws:Several anti-discrimination laws apply to ML models. These laws vary from country to country, but they generally prohibit discrimination based on race, color, religion, sex, national origin, age, disability, or other protected characteristics. For example, Title VII of the U.S. Civil Rights Act prohibits employment discrimination based on race, color, religion, sex, or national origin. In the context of ML fairness, this law ensures that algorithms and models used for employment-related decisions do not result in discriminatory outcomes.
• Data privacy regulations:Several data privacy regulations apply to ML models. These regulations vary from country to country, but they generally require that organizations obtain consent before collecting or using personal data and take steps to protect the privacy of personal data. For example, General Data Protection Regulation (GDPR), applicable in the European Union, provides guidelines for legally processing personal data. ML models handling personal data must comply with GDPR principles, such as data minimization, purpose limitation, transparency, and accountability. Individuals have the right to know how their data is used and can request explanations of automated decisions that affect them.

• Transparency and explainability:ML models should be transparent and explainable. This means that users should understand how the model works and why it makes its predictions.
• Accountability and responsibility:Organizations that develop and use ML models should be accountable for the fairness of those models. This means that they should have processes in place to identify and address bias in their models and be able to explain how they have addressed any bias.
• Inclusiveness and accessibility:ML models should be inclusive and accessible. This means they should be designed to work for people from all backgrounds and abilities.

For example, the IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems covers the guidelines:

• Advocates for transparency in designing and operating AI systems, ensuring that humans understand the basis for their decisions and actions.
•  Emphasizes the need for a clear assignment of responsibility for AI systems and encourages developers to consider the societal impact of their creations.
• Promotes the development of AI systems that are accessible, usable, and beneficial to all individuals, regardless of their abilities or backgrounds.

• Industry-specific guidelines:There are several industry-specific guidelines for ML fairness. Industry associations and organizations develop these guidelines and guide the development and using ML models fairly and responsibly. For example, AI Fairness 360 by IBM offers industry-specific guidelines to mitigate biases in AI and ML models. It provides toolkits and resources for assessing and addressing bias across healthcare, finance, and hiring domains.
• Cross-industry initiatives:Several cross-industry initiatives are working to promote ML fairness. These initiatives bring together organizations from different industries to share best practices and to develop new tools and resources to help organizations build fair and responsible ML models. For example, Partnership on AI is a consortium of technology companies and organizations collaborating to develop and share the best AI and ML fairness practices. They work on cross-industry initiatives to address fairness, ethics, and human rights challenges.
• Compliance and certification programs:There are several compliance and certification programs that organizations can participate in to demonstrate their commitment to ML fairness. These programs provide organizations with a framework for developing and using ML models fairly and responsibly.

By following these legal, ethical, and regulatory frameworks, organizations can help to ensure that their ML models are fair and equitable for all users. For example, The European Union has proposed a certification framework for AI systems, including ML models. The framework aims to assess and certify AI systems based on their compliance with legal, ethical, and technical requirements. The certification provides a recognized mark of trustworthiness and fairness.

Fairness in machine learning models is critical for building ethical and responsible AI systems. Organizations can mitigate bias by employing various strategies, such as pre-processing, in-processing, and post-processing methods. They can also follow MLOps best practices, such as continuous monitoring and evaluation, iterative model improvement, and robust model governance. Collaboration with domain experts and diverse teams is vital for understanding the societal impact of ML models and addressing fairness concerns. Interdisciplinary collaboration brings together experts from various fields to offer diverse perspectives, identify biases, and develop effective mitigation strategies. Stakeholder engagement is also crucial for ensuring that ML models align with their needs and values. As AI continues to evolve, it is imperative to prioritize fairness and mitigate biases to build trust in ML models and promote ethical decision-making. Organizations can contribute to a more inclusive and equitable future by adopting these strategies.