Scribe logoScribeNotes & practice
Sign inStart free
  1. Notes
  2. ›AP Statistics
  3. ›3 Inference for Categorical Data: Proportions
  4. ›3.8 Errors in tests

AP Statistics · Unit 3 · Inference for Categorical Data: Proportions

3.8 Errors in tests

Describe Type I and Type II errors in a hypothesis test and give the consequences of each in context.

  • 4 min read
  • 5 parts
  • 10 questions
  • Draft
Start a Quick 5 on this topic→Jump to the 10 questions
1/5A test can reach the wrong conclusion even when done correctly
  1. 1A test can reach the wrong conclusion even when done correctly
  2. 2A Type I error rejects a true null hypothesis
  3. 3A Type II error fails to detect a real effect
  4. 4Lowering one error risk usually raises the other
  5. 5The cost of each error depends on the context
  6. Check your recall
  7. Practice: 10 questions
1

Part 1 of 5: A test can reach the wrong conclusion even when done correctly

A hypothesis test uses one sample to judge a claim about a population. Samples vary from one to the next, so a sample can sometimes point the wrong way. Even a carefully run test can reach the wrong conclusion.

Every test ends with one of two decisions. We reject the null hypothesis, or we fail to reject it. The null hypothesis is the claim of no change or no difference. In reality, that claim is either true or false, and we never know which.

Crossing the two decisions with the two realities gives four outcomes. Two are correct and two are errors.

Decisions and outcomes in a hypothesis test

Null hypothesis is trueNull hypothesis is false
We reject the null hypothesisType I errorCorrect decision
We fail to reject the null hypothesisCorrect decisionType II error
We reject the null hypothesis
Null hypothesis is true
Type I error
Null hypothesis is false
Correct decision
We fail to reject the null hypothesis
Null hypothesis is true
Correct decision
Null hypothesis is false
Type II error

Part 1 done

2

Part 2 of 5: A Type I error rejects a true null hypothesis

Start with the top-left error in the table. A Type I error happens when we reject a null hypothesis that is actually true. The sample suggests a difference, but none really exists. It is a false alarm.

The probability of a Type I error equals the significance level α\alphaα, the cutoff chosen before the test. With α=0.05\alpha=0.05α=0.05, a true null hypothesis is rejected in about 5% of all possible samples. We choose α\alphaα before collecting data, so we control this risk directly.

Common mistake

The significance level is not the probability that the null hypothesis is true. It is the probability of rejecting the null hypothesis when it is true.

Key term

Type I error

A Type I error is a false positive: the null hypothesis is true but the test rejects it, with probability equal to α\alphaα.

Part 2 done

Quick check

One question on the part you just read

Easy

Identify the description of a Type I error.
Show answer and explanation

Answer: C · Worked explanation

A Type I error is a false alarm: the null hypothesis is true, yet the test rejects it. Failing to reject a false null hypothesis describes a Type II error, which is a missed detection. Rejecting a false null hypothesis and failing to reject a true one are both correct decisions.
3

Part 3 of 5: A Type II error fails to detect a real effect

The second error runs the other way. A Type II error happens when we fail to reject a null hypothesis that is actually false. A real difference exists, but our sample did not reveal it. It is a missed detection.

The probability of a Type II error is written β\betaβ. We do not choose it directly. It depends on the sample size, on α\alphaα, and on how far the true value lies from the null value.

The chance of correctly rejecting a false null hypothesis is the power of a test. Power equals 1−β1-\beta1−β. If β=0.20\beta=0.20β=0.20, the power is 0.80.

Exam tip

Write fail to reject, never accept. A Type II error means the test found no convincing evidence, not that no effect exists.

Key terms · tap one for its definition

Type II error

A Type II error is a false negative: the null hypothesis is false but the test fails to reject it, with probability β\betaβ.

power

The power of a test is the probability of rejecting the null hypothesis when a specific alternative value is true; it equals 1−β1-\beta1−β.

Part 3 done

4

Part 4 of 5: Lowering one error risk usually raises the other

It would be ideal to make both errors rare. With a fixed sample size, the two risks pull against each other.

Lowering α\alphaα makes rejection harder. Fewer true null hypotheses get rejected, so Type I errors become rarer. Fewer false null hypotheses get rejected too, so β\betaβ rises and power falls.

Three changes reduce β\betaβ and raise power:

  • A larger sample size lowers β\betaβ and leaves α\alphaα unchanged.
  • A larger α\alphaα lowers β\betaβ but raises the Type I risk.
  • A true value farther from the null value lowers β\betaβ.

Part 4 done

Quick check

One question on the part you just read

Easy

A Type II error occurs when a test
Show answer and explanation

Answer: B · Worked explanation

A Type II error is a missed detection: a real effect exists, so the null hypothesis is false, but the sample does not lead to rejecting it. Rejecting a true null hypothesis is the reverse mistake, which is a Type I error. Failing to reject a true null hypothesis is a correct decision, not an error.
5

Part 5 of 5: The cost of each error depends on the context

Now consider what these errors cost in real settings. Neither error is always the worse one. The answer depends on the harm each one causes.

Worked example

Describing both errors in context

A drug maker tests H0:p=0.70H_0: p=0.70H0​:p=0.70 against Ha:p>0.70H_a: p>0.70Ha​:p>0.70, where ppp is the true proportion of patients cured by a new drug. Describe a Type I error, a Type II error, and one consequence of each.
  1. 1
    A Type I error rejects a true null hypothesis. Here, the data suggest the cure proportion exceeds 0.70 when it is really 0.70.
  2. 2
    A Type II error fails to reject a false null hypothesis. Here, the data show no convincing improvement, but the true proportion is above 0.70.
  3. 3
    Type I consequence: patients switch to a drug that is no better, and may face extra cost or side effects.
  4. 4
    Type II consequence: patients miss a drug that would have cured more people.

Answer: Type I: concluding the new drug cures more than 70% of patients when it does not. Type II: failing to reach that conclusion when it does.

Exam tip

Always write each error in context. Name the population parameter, the true situation, and the conclusion reached.

Priorities can differ between settings. Suppose a factory tests the null hypothesis that a batch of syringes meets a safety standard. A Type II error ships an unsafe batch. A Type I error only discards a good batch. In that setting, a researcher may accept a larger α\alphaα to lower β\betaβ.

Note read

Check your recall[12 marks]

6 written questions. Answer each one in your head or on paper, then open the mark scheme.

  1. 1.Identify[1 mark]

    Identify the type of error made when a test rejects a null hypothesis that is actually true.

    Show mark schemeHide mark scheme

    Model answer

    A Type I error.

    • 1 mark for identifying a Type I error
  2. 2.Describe[2 marks]

    A bakery tests H0:p=0.10H_0: p=0.10H0​:p=0.10 against Ha:p>0.10H_a: p>0.10Ha​:p>0.10, where ppp is the true proportion of loaves with a defective crust. Describe a Type I error in this context.

    Show mark schemeHide mark scheme

    Model answer

    The test concludes that the proportion of defective loaves exceeds 0.10 when the true proportion is actually 0.10.

    • 1 mark for stating that the null hypothesis is rejected (convincing evidence that the proportion exceeds 0.10)
    • 1 mark for stating that the true proportion is actually 0.10, so the null hypothesis is true
  3. 3.Describe[2 marks]

    For the bakery test with H0:p=0.10H_0: p=0.10H0​:p=0.10 and Ha:p>0.10H_a: p>0.10Ha​:p>0.10, describe a Type II error in context.

    Show mark schemeHide mark scheme

    Model answer

    The test finds no convincing evidence that the proportion of defective loaves exceeds 0.10, but the true proportion is actually above 0.10.

    • 1 mark for stating that the null hypothesis is not rejected (no convincing evidence of a proportion above 0.10)
    • 1 mark for stating that the true proportion of defective loaves is actually above 0.10
  4. 4.Explain[2 marks]

    Explain why lowering the significance level from 0.05 to 0.01, with the same sample size, increases the probability of a Type II error.

    Show mark schemeHide mark scheme

    Model answer

    A smaller α\alphaα makes rejecting the null hypothesis harder. The test therefore also rejects fewer false null hypotheses, so β\betaβ rises.

    • 1 mark for stating that a smaller α\alphaα makes rejection harder
    • 1 mark for linking this to fewer false null hypotheses being rejected, so β\betaβ increases
  5. 5.Predict[2 marks]

    Predict how increasing the sample size, while keeping α\alphaα fixed, changes β\betaβ and the power of the test.

    Show mark schemeHide mark scheme

    Model answer

    The probability β\betaβ decreases and the power increases, while α\alphaα stays the same.

    • 1 mark for predicting that β\betaβ decreases
    • 1 mark for predicting that power increases
  6. 6.Justify[3 marks]

    A city tests the null hypothesis that a bridge is safe to carry traffic. Justify which error, Type I or Type II, has the more serious consequence.

    Show mark schemeHide mark scheme

    Model answer

    A Type I error closes a bridge that is actually safe, which costs money and causes inconvenience. A Type II error leaves an unsafe bridge open, which could cause injury or loss of life. The Type II error is more serious because the harm to people outweighs the cost of a closure.

    • 1 mark for describing a Type I error as rejecting a true null hypothesis, so a safe bridge is closed unnecessarily
    • 1 mark for describing a Type II error as failing to reject a false null hypothesis, so an unsafe bridge stays open
    • 1 mark for concluding that the Type II error is more serious because harm to people outweighs the cost of closure

Practice this topic

10 quick-fire multiple-choice questions, easy to hard. Tap an answer to see if you're right and the worked explanation. (The written questions with mark schemes are under “Check your recall” above.)

0 right0 of 10 answered

Includes the 2 quick checks in the note above.

3.

Easy·Identify[1 mark]

Identify the quantity that equals the probability of a Type I error.
Show answer and explanation

Answer: D · Worked explanation

The significance level α\alphaα is the cutoff chosen before the test, and it is the probability of rejecting a true null hypothesis. The probability β\betaβ belongs to Type II errors. The probability that the null hypothesis is true is not what α\alphaα measures, since α\alphaα is a probability of a decision, not of a hypothesis.
4.

Medium·Calculate[2 marks]

A test has a probability of a Type II error of β=0.22\beta=0.22β=0.22. Calculate the power of the test.
Show answer and explanation

Answer: C · Worked explanation

Power equals 1−β1-\beta1−β, so power =1−0.22=0.78=1-0.22=0.78=1−0.22=0.78. The value 0.22 is β\betaβ itself, the probability of missing a real effect, not the probability of detecting it. The value 1.22 comes from adding β\betaβ to 1 instead of subtracting it, and a probability cannot exceed 1.
5.

Medium·Identify[2 marks]

A city tests H0:p=0.25H_0: p=0.25H0​:p=0.25 against Ha:p<0.25H_a: p<0.25Ha​:p<0.25, where ppp is the proportion of residents who ride a bus. The test fails to reject H0H_0H0​, but the true proportion is 0.20. Identify what happened.
Show answer and explanation

Answer: B · Worked explanation

Since p=0.20p=0.20p=0.20, the null hypothesis p=0.25p=0.25p=0.25 is false. Failing to reject a false null hypothesis is a Type II error. Calling the null hypothesis true is the tempting trap: 0.20 differs from 0.25, so the null hypothesis is false, and a Type I error needs a rejection, which did not occur.
6.

Medium·Predict[2 marks]

A significance level of α=0.05\alpha=0.05α=0.05 is used on 400 independent random samples from a population where the null hypothesis is true. Predict about how many samples lead to rejecting the null hypothesis.
Show answer and explanation

Answer: B · Worked explanation

Each rejection is a Type I error because the null hypothesis is true, and these occur about 5% of the time. Then 0.05×400=200.05\times400=200.05×400=20 samples. The number 380 is the count expected to give a correct decision, 0.95×4000.95\times4000.95×400, and 5 confuses the percentage with a count.
7.

Medium·Identify[2 marks]

A company tests the null hypothesis that a machine part meets its safety standard. Identify the description of a Type I error and its consequence.
Show answer and explanation

Answer: D · Worked explanation

A Type I error rejects a true null hypothesis. Here the part really meets the standard, but the test rejects that claim, so a good part is thrown away. Judging a faulty part to meet the standard is a Type II error, because a false null hypothesis was not rejected, and it leaves a dangerous part in service.
8.

Hard·Predict[2 marks]

A researcher lowers the significance level from 0.05 to 0.01 and keeps the same sample size. Predict the effect on the two error probabilities.
Show answer and explanation

Answer: A · Worked explanation

A smaller α\alphaα makes rejection harder, so Type I errors become rarer, but false null hypotheses are also rejected less often, so β\betaβ rises. Both risks falling is the tempting error: with a fixed sample size, lowering one raises the other. Only a larger sample size lowers both.
9.

Hard[2 marks]

A test fails to reject the null hypothesis. Which statement must be true?
Show answer and explanation

Answer: C · Worked explanation

Failing to reject leaves two possibilities: the null hypothesis is true, which is a correct decision, or it is false, which is a Type II error. A Type I error needs a rejection, so it cannot occur here. We cannot conclude the null hypothesis is true, because the test may simply have missed a real effect.
10.

Hard·Compare[2 marks]

Test A and Test B have the same hypotheses and the same significance level, α=0.05\alpha=0.05α=0.05. Test A uses a sample of 200 and Test B uses a sample of 50. The true value is the same distance from the null value in both. Compare the tests.
Show answer and explanation

Answer: D · Worked explanation

Both tests have a Type I error probability of 0.05, because α\alphaα is chosen and does not depend on sample size. The larger sample in Test A lowers β\betaβ and raises power. Saying a larger sample lowers the Type I error probability confuses α\alphaα, which the researcher sets, with β\betaβ.

Keep going

Next topic3.9 Differences in sample proportionsPrevious topic3.7 Carrying out a test for a proportion

More in Unit 3 · Inference for Categorical Data: Proportions

  1. 3.4Claims from a proportion intervalNotes · 10 questions
  2. 3.5Setting up a test for a proportionNotes · 10 questions
  3. 3.6p-valuesNotes · 10 questions
  4. 3.7Carrying out a test for a proportionNotes · 10 questions
  5. 3.8Errors in testsNotes · 10 questions
  6. 3.9Differences in sample proportionsNotes · 10 questions
  7. 3.10Interval for a difference of proportionsNotes · 10 questions
  8. 3.11Claims from a difference intervalNotes · 10 questions
  9. 3.12Setting up a two-proportion testNotes · 10 questions

All AP Statistics topics →

Draft: written by Scribe and checked automatically, not yet by a subject reviewer.

Practice questions are original, written by Scribe and checked automatically for correctness.

Topic codes follow the official numbering. Not endorsed by College Board or any exam board.

On this page

0/5 read

  1. 1A test can reach the wrong conclusion even when done correctly
  2. 2A Type I error rejects a true null hypothesis
  3. 3A Type II error fails to detect a real effect
  4. 4Lowering one error risk usually raises the other
  5. 5The cost of each error depends on the context
  6. Check your recall
  7. Practice: 10 questions
All coursesScribeHow it worksPricingPrivacyTerms

Notes and questions are written by Scribe in our own words. Topic codes follow the official course numbering so you can find your place. Not produced or endorsed by the IB, College Board, AQA, OCR, Pearson Edexcel or any exam board.

© 2026 Scribe