IB Computer Science · A4 · Machine learning
A4.2 Data preprocessing
Explain why raw data is preprocessed and describe the main techniques for cleaning and transforming it.
1/5Raw data is rarely ready for a model to use
Part 1 of 5: Raw data is rarely ready for a model to use
Suppose a survey records ages, but some people typed ‘twenty’ and others left the box blank. A model cannot learn sensibly from such entries. Data preprocessing is the work of preparing raw data before training.
A model learns from whatever it is given. Poor input gives poor results, often summed up as ‘garbage in, garbage out’. Preprocessing therefore takes up much of a real project.
It has two broad parts. Cleaning fixes problems in the data. Transforming changes its form so that a model can use it.
Key term
data preprocessing
Data preprocessing is the cleaning and transforming of raw data into a form suitable for training a machine learning model.
Part 2 of 5: Cleaning removes errors and fills gaps
Start with cleaning. The first problem is a missing value, an entry with no recorded data. You can delete the affected records, or fill each gap with an estimate such as the mean of that feature.
Duplicate records give some cases extra weight, so remove them. Inconsistent entries, such as ‘UK’ and ‘United Kingdom’, should be made uniform.
An outlier is a value far from the rest. It may be an error, such as a height of 1750 cm, or a rare true case. Check it before removing it.
Key terms · tap one for its definition
missing value
A missing value is an entry in a dataset for which no data was recorded.
outlier
An outlier is a data value that lies unusually far from most other values in the dataset.

Quick check
One question on the part you just read
Easy
Show answer and explanation
Answer: A · Worked explanation
Part 3 of 5: Normalisation puts features on a common scale
Cleaning leaves valid data, but its scale may still cause trouble. Compare floor area, about 100 m², with number of rooms, about 4. A model that compares distances would be dominated by the larger numbers.
Normalisation rescales a feature to a fixed range, usually 0 to 1. Min–max scaling does this using the smallest and largest values of the feature, and : .
Worked example
Min–max scaling
- Scaled value = the value minus the smallest, divided by the range.
- Write the equation: .
- Substitute: .
- Divide: .
Answer: The scaled value is 0.5. It has no unit, because the units cancel.
The smallest value always becomes 0 and the largest becomes 1.
Key term
normalisation
Normalisation is the rescaling of a numerical feature to a fixed range, usually 0 to 1, so that features are comparable.
Part 4 of 5: Categories and unwanted features also need transforming
Scaling handles numbers, but data also holds categories such as colour or country. Most algorithms need numbers, and cannot average ‘red’ and ‘blue’. Encoding converts categories into numeric form.
One-hot encoding gives each category its own column. A 1 marks the category that applies, and 0 fills the rest.
One-hot encoding of colour
- Colour
- Red
- Red
- 1
- Green
- 0
- Blue
- 0
- Colour
- Green
- Red
- 0
- Green
- 1
- Blue
- 0
- Colour
- Blue
- Red
- 0
- Green
- 0
- Blue
- 1
Feature selection keeps useful features and drops irrelevant ones, such as a customer ID number. Fewer features can shorten training. They can also reduce overfitting, where a model learns quirks of its examples.
Key terms · tap one for its definition
encoding
Encoding is the conversion of categorical data into numeric values that an algorithm can process.
feature selection
Feature selection is the choice of the most relevant features for training, with irrelevant features removed.

Quick check
One question on the part you just read
Easy
Show answer and explanation
Answer: C · Worked explanation
Part 5 of 5: Poor data quality leads to unreliable models
Finally, consider what happens when preprocessing is skipped. Data quality describes how accurate, complete, consistent and relevant a dataset is for its purpose. Each weakness harms a model in a different way.
- Inaccurate or noisy data teaches false patterns.
- Missing or unrepresentative data leaves gaps, so the model fails on unseen cases.
- Inconsistent formats cause records to be misread or ignored.
- Unscaled features let large numbers dominate the result.
- Irrelevant features add noise and can encourage overfitting.
Apply the same steps to new data at deployment. Otherwise the model receives input in a form it never learned.
Key term
data quality
Data quality is the extent to which a dataset is accurate, complete, consistent and relevant for its intended purpose.
Check your recall[14 marks]
5 written questions. Answer each one in your head or on paper, then open the mark scheme.
- 1.Explain[3 marks]
Explain why data preprocessing is important in machine learning.
Show mark schemeHide mark scheme
Model answer
A model learns from whatever data it is given, so errors and gaps lead to poor predictions. Cleaning removes problems such as duplicates and missing values. Transforming, for example by scaling, puts the data in a form the model can use properly.
- 1 mark for stating that a model learns from the data it is given, so poor data gives poor results
- 1 mark for stating that cleaning removes errors, gaps or duplicates
- 1 mark for stating that transforming puts data in a suitable form, such as a common scale
- 2.Describe[2 marks]
Describe two ways of handling missing values in a dataset.
Show mark schemeHide mark scheme
Model answer
Delete the records that contain missing values. Alternatively, fill each gap with an estimate, such as the mean of that feature.
- 1 mark for describing deleting the affected records
- 1 mark for describing filling gaps with an estimate such as the mean
- 3.Determine[3 marks]
Determine the min–max scaled value of 20 for a feature with a smallest value of 10 and a largest value of 50.
Show mark schemeHide mark scheme
Model answer
.
- 1 mark for writing the min–max equation
- 1 mark for substituting 20, 10 and 50 correctly
- 1 mark for obtaining 0.25
- 4.Outline[2 marks]
Outline how one-hot encoding represents the colours red, green and blue.
Show mark schemeHide mark scheme
Model answer
Each colour gets its own column. A 1 is placed in the column of the colour that applies, and 0 in the other columns.
- 1 mark for stating that each category has its own column
- 1 mark for stating that 1 marks the matching category and 0 marks the others
- 5.Discuss[4 marks]
Discuss the consequences of using poor-quality data to train a model.
Show mark schemeHide mark scheme
Model answer
Inaccurate or noisy data teaches the model false patterns. Missing or unrepresentative data means it fails on cases it has not seen. Inconsistent formats lead to records being misread. Overall, predictions become unreliable and may be unfair.
- 1 mark for explaining that inaccurate or noisy data teaches false patterns
- 1 mark for explaining that missing or unrepresentative data causes failure on unseen cases
- 1 mark for stating that inconsistent formats cause records to be misread or ignored
- 1 mark for concluding that predictions become unreliable or unfair

Practice this topic
10 quick-fire multiple-choice questions, easy to hard. Tap an answer to see if you're right and the worked explanation. (The written questions with mark schemes are under “Check your recall” above.)
0 right0 of 10 answered
Includes the 2 quick checks in the note above.
Easy[1 mark]
Show answer and explanation
Answer: D · Worked explanation
Medium[1 mark]
Show answer and explanation
Answer: B · Worked explanation
Medium[1 mark]
Show answer and explanation
Answer: C · Worked explanation
Medium[1 mark]
Show answer and explanation
Answer: A · Worked explanation
Medium[1 mark]
Show answer and explanation
Answer: B · Worked explanation
Hard[2 marks]
Show answer and explanation
Answer: C · Worked explanation
Hard[2 marks]
Show answer and explanation
Answer: A · Worked explanation
Hard[2 marks]
Show answer and explanation
Answer: B · Worked explanation