Skip to Main Content

AI Resources and Support

Equity Issues with AI

AI, Accessibility, and Equity 

Consider these reflections on the potential of AI to impact access, inclusion, and equity in higher education. 

The Impact of AI on Accessibility (ER Podcast)

Algorithmic Bias

Bias in AI

Introduction

The human brain is an intricate organ that functions through conscious and unconscious connections. While we may view explicit bias as ethically incorrect, our brains automatically establish implicit associations. This is not within our control but rather a natural function of our brains.

When it comes to generative AI, it is essential to acknowledge how these unconscious associations can affect the model and result in biased outputs. By identifying the sources of bias and creating inclusive prompts, we can strive towards developing AI systems that are more fair and just.

Stages Where Bias Can Occur

It is important to recognize that bias can occur in various stages of the AI pipeline. One of the primary sources of such bias is data collection. The resulting outputs may be biased if the data used to train an AI algorithm is not diverse or representative.

  • Data Collection: Bias often originates here. The AI algorithm might produce biased outputs if the data is not diverse or representative.
  • Data Labeling: This can introduce bias if the annotators have different interpretations of the same label.
  • Model Training: A critical phase; if the training data is not balanced or the model architecture is not designed to handle diverse inputs, the model may produce biased outputs.
  • Deployment: This can also introduce bias if the system is not tested with diverse inputs or monitored for bias after deployment.

Implicit and Explicit Bias

It is important to differentiate between explicit and implicit biases. Explicit biases are a person's conscious prejudices or beliefs towards a specific group. On the other hand, implicit biases are unconscious attitudes or stereotypes that affect our understanding, actions, and decisions without our awareness. These biases result from in-group preferences and reflect broader structural inequities. We often unknowingly absorb these biases from the dominant social messages we encounter.

Implicit biases can be particularly harmful as they are deeply ingrained and can influence our behavior even when we consciously reject them. To reduce the impact of bias and promote a culture of inclusivity and respect, we should be mindful of our language and actively seek out diverse perspectives.

Types of Bias in AI

Biases can lead to severe repercussions, especially when they contribute to social injustice or discrimination. This is because biased data can strengthen and worsen existing prejudices, resulting in systemic inequalities. Hence, it is crucial to stay alert in detecting and rectifying biases in data and models and aim for fairness and impartiality in all data-driven decision-making processes.

  • Selection bias: This happens when the data used to train an AI system is not representative of the reality it's meant to model. It can occur due to various reasons, such as incomplete data, biased sampling, or other factors that may lead to an unrepresentative dataset. If a model is trained on a dataset that only includes male employees, for example, it will not be able to predict female employees' performance accurately.
  • Confirmation bias: This type of bias happens when an AI system is tuned to rely too much on pre-existing beliefs or trends in the data. This can reinforce existing biases and fail to identify new patterns or trends.
  • Measurement bias: This bias occurs when the data collected differs systematically from the actual variables of interest. For instance, if a model is trained to predict students' success in an online course, but the data collected is only from students who have completed the course, the model may not accurately predict the performance of students who drop out of the course.
  • Stereotyping bias: This happens when an AI system reinforces harmful stereotypes. An example is when a facial recognition system is less accurate in identifying people of color or when a language translation system associates certain languages with certain genders or stereotypes.
  • Out-group homogeneity bias: When this happens, an AI system is less capable of distinguishing between individuals who are not part of the majority group in the training data; it's a form of out-group homogeneity bias. This may result in misclassification or inaccuracy when dealing with minoritized populations.

Adapted from Chapman University's AI Hub

How does algorithmic bias impact different AI applications?