Scribe logoScribeNotes & practice
Sign inStart free
  1. Notes
  2. ›AP Statistics
  3. ›1 Exploring One-Variable Data and Collecting Data
  4. ›1.12 Problems with sampling

AP Statistics · Unit 1 · Exploring One-Variable Data and Collecting Data

1.12 Problems with sampling

Identify sources of bias in sampling, explain how undercoverage and nonresponse affect results, and describe ways to reduce sampling problems.

  • 4 min read
  • 5 parts
  • 10 questions
  • Draft
Start a Quick 5 on this topic→Jump to the 10 questions
1/5Bias is a systematic error that a larger sample cannot fix
  1. 1Bias is a systematic error that a larger sample cannot fix
  2. 2Undercoverage leaves some individuals with no chance to appear
  3. 3Nonresponse and voluntary response both distort who is counted
  4. 4Question wording and honesty can bias the answers too
  5. 5Careful design reduces bias, and larger samples reduce variability
  6. Check your recall
  7. Practice: 10 questions
1

Part 1 of 5: Bias is a systematic error that a larger sample cannot fix

Even a well-planned sample can give misleading results. Two different problems need to be kept apart. The first is bias, which pushes results in the same direction every time. Mixing the two up is a common source of errors.

The second is sampling variability, the natural variation in a statistic from one random sample to the next. Variability is not systematic. It shrinks as the sample size grows.

Figure 1. A larger sample tightens the results, but biased results stay off the true value.

Common mistake

A bigger sample reduces variability, but it does not remove bias. A large biased sample is still wrong in the same direction.

Key terms · tap one for its definition

bias

Bias is a systematic tendency of a sampling method to over- or underrepresent part of the population.

sampling variability

Sampling variability is the natural variation of a statistic from one random sample to another, which decreases as the sample size increases.

Part 1 done

2

Part 2 of 5: Undercoverage leaves some individuals with no chance to appear

One source of bias comes from the list used to choose the sample. Undercoverage occurs when some members of the population have no chance of being selected. This usually happens because the list leaves them out. A survey that uses only a landline phone directory misses households that have only cell phones.

Those missing households may differ from the rest in age, income, or habits. The results then describe the people who could be reached, not the whole population.

Exam tip

Name the group that is left out, and say how that could push the results up or down.

Key term

undercoverage

Undercoverage is a source of bias in which some members of the population are left out of the sampling frame and cannot be selected.

Part 2 done

Quick check

One question on the part you just read

Easy

Identify the description of undercoverage.
Show answer and explanation

Answer: A · Worked explanation

Undercoverage means part of the population is left out of the sampling frame and cannot be selected. Refusing to answer is nonresponse, choosing whether to respond is voluntary response bias, and a question that pushes people toward one answer causes response bias.
3

Part 3 of 5: Nonresponse and voluntary response both distort who is counted

Even when the list is complete, not everyone chosen takes part. Nonresponse occurs when a selected individual cannot be contacted or refuses to answer. If those who do not respond differ from those who do, the results are biased. A low response rate makes this more likely.

Voluntary response bias occurs when individuals decide for themselves whether to respond. People with strong opinions are more likely to answer, so results tend to be extreme.

Convenience sampling, which means choosing the individuals who are easiest to reach, causes similar trouble. It leaves out people who are harder to reach.

Worked example

Identifying bias in a survey

A cafeteria manager puts a comment box near the exit and asks students to drop in notes about the food. Identify the bias and describe its likely effect.
  1. 1
    Ask who decided to take part: the students chose for themselves.
  2. 2
    Name the bias: this is voluntary response bias.
  3. 3
    Describe the effect in context: students with strong feelings, very pleased or very upset, are most likely to write.
  4. 4
    Conclude that the comments probably overrepresent extreme opinions about the food.

Answer: The bias is voluntary response bias, and the comments likely overrepresent strongly positive or strongly negative opinions.

Key terms · tap one for its definition

nonresponse

Nonresponse is a source of bias in which a selected individual cannot be reached or declines to provide data.

voluntary response bias

Voluntary response bias is bias that arises because individuals choose whether to respond, so people with strong opinions tend to be overrepresented.

Part 3 done

4

Part 4 of 5: Question wording and honesty can bias the answers too

Bias can also enter after the sample is chosen. Response bias occurs when answers are systematically inaccurate. A leading question pushes people toward one answer. Asking “Don’t you agree that homework is too long?” invites agreement.

Sensitive questions cause the same problem. People may understate behaviors that others might disapprove of. Neutral wording, private answers, and anonymous surveys can reduce this.

Even honest answers can be affected by how a question is asked. The order of the questions and the tone of an interviewer can also change what people say.

Key term

response bias

Response bias is a systematic pattern of inaccurate answers caused by question wording, sensitive topics, or the way a question is asked.

Part 4 done

Quick check

One question on the part you just read

Easy

Identify the change that reduces sampling variability.
Show answer and explanation

Answer: A · Worked explanation

A larger random sample makes results from different samples closer together, which reduces variability. Neutral wording reduces response bias, following up reduces nonresponse, and a better sampling frame reduces undercoverage. Those three changes address bias rather than variability, so they would not make results from different random samples agree more closely.
5

Part 5 of 5: Careful design reduces bias, and larger samples reduce variability

Now that you can spot each problem, consider how to reduce it. The table matches each problem with a fix.

Problems and ways to reduce them

ProblemWay to reduce it
UndercoverageUse a sampling frame that covers the whole population
NonresponseFollow up with those who do not reply
Voluntary responseSelect individuals by random sampling
Convenience samplingReplace easy-to-reach choices with random selection
Response biasUse neutral wording and private answers
Sampling variabilityTake a larger random sample
Problem
Undercoverage
Way to reduce it
Use a sampling frame that covers the whole population
Problem
Nonresponse
Way to reduce it
Follow up with those who do not reply
Problem
Voluntary response
Way to reduce it
Select individuals by random sampling
Problem
Convenience sampling
Way to reduce it
Replace easy-to-reach choices with random selection
Problem
Response bias
Way to reduce it
Use neutral wording and private answers
Problem
Sampling variability
Way to reduce it
Take a larger random sample

Only the last fix uses sample size. Replacing a nonresponder with an easy substitute does not help. The substitute may differ from the person who did not respond. Careful planning is cheaper than trying to repair a flawed sample afterward.

Exam tip

When you describe bias, name the type and say who is over- or underrepresented. Then state the likely effect on the result in context.

Note read

Check your recall[12 marks]

5 written questions. Answer each one in your head or on paper, then open the mark scheme.

  1. 1.Identify[1 mark]

    A website asks visitors to vote on a local issue, and only those who choose to vote are counted. Identify the type of bias.

    Show mark schemeHide mark scheme

    Model answer

    This is voluntary response bias.

    • 1 mark for identifying voluntary response bias
  2. 2.Explain[3 marks]

    Explain how undercoverage can lead to results that do not represent the population.

    Show mark schemeHide mark scheme

    Model answer

    Some members of the population are missing from the sampling frame, so they have no chance of being selected. The people left out may differ from those included. The results therefore systematically misrepresent the population.

    • 1 mark for stating that some members have no chance of being selected
    • 1 mark for stating that those left out may differ from those included
    • 1 mark for concluding that the results systematically misrepresent the population
  3. 3.Describe[2 marks]

    Describe two ways to reduce bias caused by nonresponse.

    Show mark schemeHide mark scheme

    Model answer

    Follow up with selected individuals who do not reply, such as by contacting them again. Make the survey easy to complete, so more selected individuals take part.

    • 1 mark for describing following up with those who did not respond
    • 1 mark for describing a second reasonable method, such as making the survey easy to complete
  4. 4.Justify[4 marks]

    A polling group has a biased method and increases its sample size from 500 to 5,000. Justify whether this removes the bias.

    Show mark schemeHide mark scheme

    Model answer

    It does not remove the bias. Bias is systematic, so it pushes results in the same direction. A larger sample only reduces sampling variability. The same part of the population is still over- or underrepresented, so the method must be changed.

    • 1 mark for stating that the bias is not removed
    • 1 mark for stating that bias is systematic and pushes results in one direction
    • 1 mark for stating that a larger sample reduces only sampling variability
    • 1 mark for stating that the method itself must change
  5. 5.Identify[2 marks]

    A survey asks, “Don’t you agree that the cafeteria food is terrible?” Identify the source of bias and describe how to reduce it.

    Show mark schemeHide mark scheme

    Model answer

    This is response bias caused by a leading question. Reword the question so it is neutral.

    • 1 mark for identifying response bias from a leading question
    • 1 mark for describing neutral wording as the fix

Practice this topic

10 quick-fire multiple-choice questions, easy to hard. Tap an answer to see if you're right and the worked explanation. (The written questions with mark schemes are under “Check your recall” above.)

0 right0 of 10 answered

Includes the 2 quick checks in the note above.

3.

Easy·Identify[1 mark]

A radio show asks listeners to call in with their opinions on a new law. Identify the main problem with this method.
Show answer and explanation

Answer: A · Worked explanation

Listeners decide for themselves whether to call, and people with strong opinions are more likely to do so. That is voluntary response bias. Undercoverage involves a list that omits people, and nothing in this scenario suggests a leading question or a problem with the list of listeners.
4.

Medium·Identify[1 mark]

A survey of 2,000 households is drawn only from a landline-only phone directory. Identify the main source of bias.
Show answer and explanation

Answer: A · Worked explanation

A landline directory leaves out households that have no landline, so those households have no chance of being selected, which is undercoverage. Nothing says that selected households refused or chose themselves, and 2,000 households would not remove a bias from the list.
5.

Medium·Calculate[1 mark]

A survey is mailed to 800 people, and 200 return it. What percentage of the selected people did not respond?
Show answer and explanation

Answer: A · Worked explanation

The number who did not respond is 800−200=600800-200=600800−200=600. The percentage is 600800×100%=75%\frac{600}{800}\times100\%=75\%800600​×100%=75%. The value 25% is the percentage who did respond. The value 33% comes from dividing 200 by 600, which compares responders with nonresponders instead of using all 800 selected people.
6.

Medium·Identify[1 mark]

A poll asks, “Wouldn’t you agree that our town needs more parks?” Identify the source of bias.
Show answer and explanation

Answer: A · Worked explanation

The wording invites agreement, so the answers are pushed in one direction, which is response bias. Undercoverage depends on who is on the list, nonresponse depends on who fails to answer, and voluntary response depends on who chooses to take part, none of which the question’s wording changes.
7.

Medium·Identify[1 mark]

Selected people often do not reply to a survey. Which action best reduces the effect of nonresponse?
Show answer and explanation

Answer: A · Worked explanation

Repeated contact attempts bring in more of the people who were actually selected, which makes responders resemble the sample more closely. A larger sample does not fix the bias, volunteers differ from the selected people, and a longer survey would likely lower the response rate.
8.

Hard·Justify[2 marks]

A website poll with 10,000 voluntary responses shows 70% support for a plan. A random sample of 400 residents shows 52% support. Which result is more trustworthy?
Show answer and explanation

Answer: A · Worked explanation

Random selection avoids the systematic bias of self-selection, so the smaller sample is more trustworthy. Size reduces variability but does not remove bias, so the 10,000 responses can still be wrong in the same direction. The two methods do not share the same bias.
9.

Hard·Predict[2 marks]

To estimate the mean weekly exercise time of all students at a school, a researcher picks 50 students at random from those present at the school gym on a Saturday morning. Which statement is best?
Show answer and explanation

Answer: A · Worked explanation

Students who rarely exercise are unlikely to be at the gym, so they are underrepresented, which is a form of undercoverage. Using chance within the gym does not fix that, because the frame is wrong. A bigger sample taken from the same gym keeps the same bias.
10.

Hard·Determine[2 marks]

A school emails a survey to 4,500 families, and 18% respond. Which statement is correct?
Show answer and explanation

Answer: A · Worked explanation

The number of responders is 0.18×4500=8100.18\times4500=8100.18×4500=810, so 4500−810=36904500-810=36904500−810=3690 did not respond. If nonresponders differ from responders, the results are biased, however large 810 seems. The value 3,690 is the number of nonresponders, not responders, and nonresponse is a source of bias, not only of variability.

Keep going

Next topic1.13 Experimental designPrevious topic1.11 Random sampling

More in Unit 1 · Exploring One-Variable Data and Collecting Data

  1. 1.5One quantitative variable: graphsNotes · 10 questions
  2. 1.6Describing a distributionNotes · 10 questions
  3. 1.7Summary statisticsNotes · 10 questions
  4. 1.8Boxplots and summary graphsNotes · 10 questions
  5. 1.9Comparing distributionsNotes · 10 questions
  6. 1.10Investigative questions and data collectionNotes · 10 questions
  7. 1.11Random samplingNotes · 10 questions
  8. 1.12Problems with samplingNotes · 10 questions
  9. 1.13Experimental designNotes · 10 questions

All AP Statistics topics →

Draft: written by Scribe and checked automatically, not yet by a subject reviewer.

Practice questions are original, written by Scribe and checked automatically for correctness.

Topic codes follow the official numbering. Not endorsed by College Board or any exam board.

On this page

0/5 read

  1. 1Bias is a systematic error that a larger sample cannot fix
  2. 2Undercoverage leaves some individuals with no chance to appear
  3. 3Nonresponse and voluntary response both distort who is counted
  4. 4Question wording and honesty can bias the answers too
  5. 5Careful design reduces bias, and larger samples reduce variability
  6. Check your recall
  7. Practice: 10 questions
All coursesScribeHow it worksPricingPrivacyTerms

Notes and questions are written by Scribe in our own words. Topic codes follow the official course numbering so you can find your place. Not produced or endorsed by the IB, College Board, AQA, OCR, Pearson Edexcel or any exam board.

© 2026 Scribe