IB Computer Science · A4 · Machine learning
A4.4 Ethical considerations
Discuss bias, privacy and wider social impact in machine learning, and explain how automated decisions affect individuals.
1/6Biased data produces biased machine learning systems
- Biased data produces biased machine learning systems
- Bias can enter at several stages of a project
- Collecting personal data raises privacy concerns
- Automated decisions can seriously affect individuals
- Machine learning also changes society as a whole
- Reducing harm needs action throughout a project
- Check your recall
- Practice: 10 questions
Part 1 of 6: Biased data produces biased machine learning systems
A model learns whatever patterns its data holds, including unfair ones. In machine learning, bias leads to results that are systematically less fair for some people or groups.
Suppose past hiring records favoured one group. A model trained on them may repeat that pattern. The model holds no prejudice of its own. It copies the patterns in its data.
Key term
bias
Bias is a systematic error in a machine learning system that produces unfair outcomes for particular individuals or groups.
Part 2 of 6: Bias can enter at several stages of a project
Bias is not only about old records. It can enter at several points in the workflow.
- Data collection: some groups are under-represented or poorly measured.
- Labelling: human labellers bring their own judgements.
- Design: the choice of features or target can favour some outcomes.
- Use: a model applied to people unlike its training data performs worse.
Accuracy is usually highest for groups the model saw most often. A face recognition system trained mainly on one group may misidentify others more often. The remedy starts with representative data.
Key term
representative data
Representative data is data that reflects the range of people and cases a model will meet when it is used.

Quick check
One question on the part you just read
Easy
Show answer and explanation
Answer: A · Worked explanation
Part 3 of 6: Collecting personal data raises privacy concerns
Bias concerns fairness. Privacy concerns what is collected about people and how it is used. Training data often holds personal data, such as browsing history, location or health records.
A key principle is informed consent. People may not know that their data is collected or that it trains a model. Further concerns follow.
- Purpose: data collected for one reason should not quietly serve another.
- Security: stored data can be leaked or stolen.
- Retention: data should not be kept longer than needed.
- Re-identification: combining datasets can identify people even after names are removed.
Key terms · tap one for its definition
personal data
Personal data is information that identifies, or can be linked to, a particular living individual.
informed consent
Informed consent is a person’s agreement to the use of their data after being told clearly how it will be used.
Part 4 of 6: Automated decisions can seriously affect individuals
Now consider the person on the receiving end. Models help decide who gets a loan, a job interview or medical attention. A wrong decision can cost someone money, opportunity or health.
Two problems make this harder. Many models are difficult to explain, so a refused applicant may never learn why. This lack of transparency makes errors hard to challenge.
There is also the question of accountability. When a model causes harm, someone must answer for it, whether developer, organisation or both.
Key terms · tap one for its definition
transparency
Transparency is the extent to which it is possible to understand how a system works and why it produced a decision.
accountability
Accountability is the responsibility of identifiable people or organisations for the outcomes of a system they build or use.

Quick check
One question on the part you just read
Easy
Show answer and explanation
Answer: B · Worked explanation
Part 5 of 6: Machine learning also changes society as a whole
Beyond individual cases, machine learning changes how society works. Its effects are mixed, with real benefits and real risks. A balanced answer gives both. The table shows four areas where each side appears.
Benefits and risks by area
- Area
- Healthcare
- Possible benefit
- Earlier detection of illness
- Possible risk
- Unequal accuracy across groups
- Area
- Employment
- Possible benefit
- Routine tasks automated
- Possible risk
- Job losses for some workers
- Area
- Information
- Possible benefit
- Content matched to interests
- Possible risk
- Narrow views and false material
- Area
- Surveillance
- Possible benefit
- Faster identification of suspects
- Possible risk
- Loss of privacy, wrongful matches
The benefits may not reach everyone equally. Groups with less data or less access may gain least. Systems used at scale also multiply any error, because one flawed model can affect very many people at once. Trust suffers if people see systems as unfair or secret. Public confidence matters for adoption.
Part 6 of 6: Reducing harm needs action throughout a project
What can developers do about these problems? Many harms can be reduced by action at each stage of the workflow.
- Use representative data and check it for gaps.
- Test accuracy separately for different groups.
- Collect only the data that is needed.
- Keep a person able to review important decisions.
- Explain how the system works and who is responsible.
Fairness is not a single measure, and trade-offs exist. Improving overall accuracy may reduce accuracy for a small group.
Check your recall[14 marks]
5 written questions. Answer each one in your head or on paper, then open the mark scheme.
- 1.Describe[3 marks]
Describe how biased training data can lead to an unfair machine learning system.
Show mark schemeHide mark scheme
Model answer
Training data may reflect past unfair patterns or under-represent some groups. The model learns these patterns from its data. It then gives less accurate or less fair results for those groups.
- 1 mark for stating that the data reflects unfair patterns or under-represents some groups
- 1 mark for stating that the model learns these patterns
- 1 mark for stating that the outcomes are less fair or less accurate for particular groups
- 2.State[2 marks]
State two privacy concerns related to the use of personal data in machine learning.
Show mark schemeHide mark scheme
Model answer
People may not have given informed consent. Stored data may be leaked or kept longer than needed.
- 1 mark for stating one valid privacy concern
- 1 mark for stating a second valid privacy concern, such as purpose, security, retention or re-identification
- 3.Explain[3 marks]
Explain why lack of transparency is a problem when a model refuses someone a loan.
Show mark schemeHide mark scheme
Model answer
The applicant cannot learn why they were refused. Without a reason, it is hard to challenge or correct an error. It also makes it unclear who is accountable for the harm.
- 1 mark for stating that the person cannot learn the reason for the decision
- 1 mark for explaining that errors are hard to challenge or correct
- 1 mark for linking this to unclear accountability for harm
- 4.Outline[2 marks]
Outline one way to reduce bias in a machine learning system.
Show mark schemeHide mark scheme
Model answer
Use representative data that covers the people the model will meet. Then test accuracy separately for each group to find unfair differences.
- 1 mark for stating a valid action, such as using representative data or testing groups separately
- 1 mark for outlining how the action reduces bias or reveals it
- 5.Discuss[4 marks]
Discuss the impact of machine learning on society, giving one benefit and one risk.
Show mark schemeHide mark scheme
Model answer
Machine learning can detect illness earlier, which benefits patients. It can also automate routine tasks, which may cause job losses. Benefits may not be shared equally, so the overall impact depends on how systems are designed and used.
- 1 mark for identifying a valid benefit
- 1 mark for identifying a valid risk
- 1 mark for explaining the impact of at least one of them
- 1 mark for reaching a balanced conclusion about the overall impact

Practice this topic
10 quick-fire multiple-choice questions, easy to hard. Tap an answer to see if you're right and the worked explanation. (The written questions with mark schemes are under “Check your recall” above.)
0 right0 of 10 answered
Includes the 2 quick checks in the note above.
EasyState[1 mark]
Show answer and explanation
Answer: C · Worked explanation
Medium[1 mark]
Show answer and explanation
Answer: D · Worked explanation
Medium[1 mark]
Show answer and explanation
Answer: A · Worked explanation
Medium[1 mark]
Show answer and explanation
Answer: B · Worked explanation
Medium[1 mark]
Show answer and explanation
Answer: C · Worked explanation
Hard[2 marks]
Show answer and explanation
Answer: A · Worked explanation
Hard[2 marks]
Show answer and explanation
Answer: B · Worked explanation
Hard[2 marks]
Show answer and explanation
Answer: D · Worked explanation