Ethics of machine learning

A model that decides who gets a loan, a job interview or a medical scan has real consequences for real people. Machine learning is powerful because it finds patterns in data without being told the rules. That is also its risk: it can find patterns we would not endorse, apply them to millions of people, and do so in a way nobody can easily inspect. This page covers the main ethical issues. As always with ethics, the aim is not to learn a list of “right answers” but to weigh benefits and harms for everyone affected, and to reach a conclusion you can justify.

These issues are connected. A non-transparent model makes bias harder to find; a lack of accountability makes harm harder to put right; collecting more data to improve fairness can threaten privacy.

Bias means systematic unfairness for or against a person or group. A model has no opinions, but it copies its data, so bias gets in through the data. There are several routes:

• History is recorded. If a bank once refused loans in a particular neighbourhood, that decision sits in the records, and a model trained on them learns to refuse that neighbourhood.
• Some people are missing. A diagnostic tool trained mainly on data from one group may be less accurate for others, leading to misdiagnosis.
• Stereotypes in text. A language model trained on biased text can repeat the stereotypes in it.
• It can be subtle. Even apparently neutral data, such as news articles, can carry the viewpoints of those who wrote it.

Real cases make the point. One large online retailer is reported to have stopped using a recruitment tool after it was found to rate women’s applications lower because of the historical hiring data. A risk-scoring tool used in some US courts was found in a widely cited analysis to label one group of defendants as high risk who did not go on to reoffend more often than another. Credit-scoring systems trained on historical financial data can reproduce unequal access to loans.

Three things to take from the lab. First, a model told the group and trained to match past decisions copies the bias. Second, when the group is hidden but home area is available, the area stands in for it (a proxy variable) and much of the unfairness comes back. Third, it is fair to ask what “fair” should mean: matching past decisions is not the same as treating equally qualified people equally.

Examples make an exam answer concrete. These are cases reported in the textbooks. Describe them as reported, and use them to illustrate an issue rather than as proof of a general rule.

Security deserves its own mention, because a machine learning system can be attacked in ways that ordinary software cannot. Data poisoning puts bad examples into the training data to steer the model. Model evasion uses crafted inputs (for example tricky prompts) to make a trained model give wrong or forbidden answers. Model inversion tries to extract sensitive training data from the model. A system that is not secure cannot be fair or private either.

• Misinformation and disinformation. Models can generate and spread false content at scale. A deepfake can make it appear that someone said or did something they did not, and has been used for fraud.
• Harassment and hate speech. Models can detect and filter harmful content, but bots can also be used to automate abuse, and filters can wrongly silence legitimate voices.
• Bias in recommendation and moderation. Systems that choose what you see or what is removed can treat groups unequally and shape what people believe. A system built to maximise engagement tends to recommend more of what you already like, which can create an echo chamber that hardens opinions.
• Anonymity and accountability. Anonymity protects vulnerable people but can also shield abuse. Tools to identify users raise surveillance concerns.
• Privacy. Targeted advertising relies on detailed tracking of online behaviour, and models can infer things people never told them, such as a health condition. Data that is meant to be anonymous can sometimes be matched with other data and re-identified.

No single fix works, so the approach is several layers together.

• Better data: representative, diverse and checked for bias; personal data sanitised and anonymised before use.
• Testing: measure results for different groups before and after launch (not just overall accuracy), and keep monitoring.
• Diverse teams who are more likely to notice problems that others miss.
• Consent and transparency: tell people what is collected and how decisions are made; give them a way to challenge a decision.
• Human oversight for decisions that matter, so a person remains accountable.
• Security against attack and misuse, and regulation that sets minimum standards.