Direct discrimination is when somebody is disadvantaged because of a personal attribute (e.g. age, gender, race).
Indirect discrimination is when people with a certain personal attribute (e.g. age, gender, race) are disadvantaged even though the attribute is not explicitly considered in decisions.
Bias in the context of AI ethics looks at:
- imbalance or tendency in data (input)
- direct or indirect discrimination (output)
Examples of bias
- A U.S. court trying to predict reoffenders were biased against black people.
- Pedestrian detection miss rates on children.
- Gender classification error rates based on race.
Causes of discrimination by AI
- Risk is not anticipated, tested, or alleviated
- “world bias”: world distribution problem
- “representation bias”: not enough data is collected to accurately represent
- “measurement bias”: wrong categorisation of people / wrong measurements
- “algorithm bias”: wrong choice of algorithm
- “evaluation bias”: wrong choice of evaluation metric or test set
- Risk is obvious, problematic task
- Ethics board does not flag a problem
- Developer doesn’t oppose to build, doesn’t report, doesn’t blow whistle
- Management makes decision to deploy