Questions on Type I and Type II errors, test size and power

Question 1

Define a Type I error and a Type II error.

Accepted Answer

A Type I error rejects H₀ when H₀ is true: a false alarm. A Type II error fails to reject H₀ when a particular alternative is true: a missed change. B1 B1 for the two definitions. The first needs no alternative specified; the second cannot be computed without one.

Question 2

For X ~ B(10, p) with H₀: p = 0.5, H₁: p < 0.5 and critical region X ≤ 1, find the size of the test.

Accepted Answer

Size = P(X ≤ 1 | p = 0.5) = (1 + 10)/2¹⁰ = 11/1024 = 0.0107. M1 for the lower tail under H₀, A1 for 11/1024, A1 for 0.0107. That sits well below a nominal 5%. The distribution is discrete, so the jump from X ≤ 1 to X ≤ 2 overshoots the level and no region has size exactly 0.05.

Question 3

Define the size and the power of a hypothesis test.

Accepted Answer

The size is the probability of rejecting H₀ when H₀ is true, that is, the probability of a Type I error. The power is the probability of rejecting H₀ when a specified alternative value of the parameter is true, which is 1 minus the probability of a Type II error at that value. B1 B1 for the two definitions. Both are probabilities of the same event, rejection, computed under different values of the parameter.

Question 4

A test of H₀: λ = 3 against H₁: λ < 3 uses a single observation X ~ Po(λ) with critical region X = 0. Find the size of the test and its power when λ = 1.

Accepted Answer

Size = P(X = 0 | λ = 3) = e⁻³ = 0.0498. Power at λ = 1 is P(X = 0 | λ = 1) = e⁻¹ = 0.3679. M1 for P(X = 0) under H₀, A1 for 0.0498, B1 for 0.3679. A fall from 3 to 1 is detected only about 37% of the time by one observation.

Question 5

A test of H₀: p = 0.5 against H₁: p < 0.5 uses X ~ B(10, p) with critical region X ≤ 1. Find the probability of a Type II error when p = 0.2, and the power at that value.

Accepted Answer

P(X ≤ 1 | p = 0.2) = 0.8¹⁰ + 10(0.2)(0.8⁹) = 0.1074 + 0.2684 = 0.3758, so the test rejects with probability 0.3758: that is the power. The Type II probability is 1 − 0.3758 = 0.6242, so the test misses a drop to 0.2 nearly two times in three. M1 for the binomial terms, A1 for 0.3758, A1 for naming that as the power, A1 for 0.6242.

Question 6

A test of H₀: p = 0.5 against H₁: p < 0.5 uses X ~ B(10, p) with critical region X ≤ 1, and its power at p = 0.2 is 0.376. Find the power at p = 0.1 and comment on how the two compare.

Accepted Answer

P(X ≤ 1 | p = 0.1) = 0.9¹⁰ + 10(0.1)(0.9⁹) = 0.3487 + 0.3874 = 0.7361. M1 for the two binomial terms, A1 for 0.7361, B1 for the comparison. The power has almost doubled: the further the truth lies from H₀, the more easily the test detects it.

Question 7

A Poisson test has H₀: λ = 5 against H₁: λ > 5 with critical region X ≥ 10. Find its size, and its power when λ = 8.

Accepted Answer

Size = P(X ≥ 10 | λ = 5) = 1 − P(X ≤ 9) = 1 − 0.9682 = 0.0318. Power at λ = 8 is P(X ≥ 10 | λ = 8) = 1 − P(X ≤ 9) = 1 − 0.7166 = 0.2834. M1 for the one minus form, A1 for 0.0318, M1 for the same at λ = 8, A1 for 0.2834. A rise from 5 to 8 is caught barely more than a quarter of the time by a single observation.

Question 8

Explain why widening a critical region raises the power of a test but is not automatically an improvement.

Accepted Answer

A wider region rejects H₀ more readily, so it catches more true departures and the power rises at every alternative. It also rejects more often when H₀ is true, so the size rises with it and there are more false alarms. With the sample size fixed the two move together, and the right balance depends on which error costs more. B1 for the power rising, B1 for the size rising, B1 for the trade-off. Only a larger sample improves both at once.

Question 9

X ~ B(20, p) is used to test H₀: p = 0.3 against H₁: p < 0.3 with critical region X ≤ 2. Write down the power function of the test, evaluate it at p = 0.2 and at p = 0.05, and interpret the two values.

Accepted Answer

Power(p) = P(X ≤ 2 | p) = (1 − p)²⁰ + 20p(1 − p)¹⁹ + 190p²(1 − p)¹⁸.
At p = 0.2: 0.8²⁰ + 20(0.2)(0.8¹⁹) + 190(0.04)(0.8¹⁸) = 0.0115 + 0.0576 + 0.1369 = 0.2061. At p = 0.05: 0.95²⁰ + 20(0.05)(0.95¹⁹) + 190(0.0025)(0.95¹⁸) = 0.3585 + 0.3774 + 0.1887 = 0.9245.
A fall from 0.3 to 0.2 is detected only about one time in five, so the test is weak against a modest change; a fall to 0.05 is detected more than nine times in ten. B1 for the power function, M1 for evaluating the three terms, A1 for both values, B1 for the interpretation.

Question 10

A single observation X is taken from N(m, 16), where m is unknown. To test H₀: m = 50 against H₁: m > 50, H₀ is rejected if X > 56. Find the size of the test, and the probability of a Type II error when m = 58.

Accepted Answer

Size = P(X > 56 | m = 50) = P(Z > (56 − 50)/4) = P(Z > 1.5) = 1 − 0.9332 = 0.0668. A Type II error is failing to reject when m = 58: P(X ≤ 56 | m = 58) = P(Z ≤ (56 − 58)/4) = P(Z ≤ −0.5) = 0.3085. M1 for standardising 56 with m = 50, A1 for 0.0668, M1 for standardising 56 with m = 58 and taking the lower tail, A1 for 0.3085.

Question 11

A test of H₀: p = 0.5 against H₁: p < 0.5 uses X ~ B(10, p) with critical region X ≤ 1, which has size 0.0107 and power 0.3758 at p = 0.2. A second test uses X ~ B(20, p) with critical region X ≤ 5. Find the size of the second test and its power at p = 0.2, and compare the two tests.

Accepted Answer

Size = P(X ≤ 5 | p = 0.5) = 0.0207, from the cumulative binomial table for B(20, 0.5). Power at p = 0.2 = P(X ≤ 5 | p = 0.2) = 0.8042. Doubling the sample roughly doubles the size, from 0.0107 to 0.0207, but both are below 5%; the power more than doubles, from 0.3758 to 0.8042. The second test detects a fall to 0.2 four times in five where the first managed fewer than two in five, at a false-alarm rate that is still small. M1 for reading the lower tail at p = 0.5, A1 for 0.0207, A1 for 0.8042, B1 for the comparison.

Question 12

X ~ B(20, p) is used to test H₀: p = 0.3 against H₁: p < 0.3 at the 5% level. Given P(X ≤ 2 | p = 0.3) = 0.0355 and P(X ≤ 3 | p = 0.3) = 0.1071, find the critical region and the size of the test, then find the power at p = 0.15 and the probability of a Type II error there.

Accepted Answer

The alternative is one-tailed and low, so the critical region sits in the lower tail. P(X ≤ 3) = 0.1071 exceeds 0.05, so 3 cannot be included, while P(X ≤ 2) = 0.0355 does not. The critical region is X ≤ 2.
The size is the probability of rejecting H₀ when it is true, that is 0.0355, not the nominal 0.05. A discrete distribution rarely offers a region of size exactly 0.05, and the actual size is what must be quoted.
Power at p = 0.15 = P(X ≤ 2 | p = 0.15) = 0.85²⁰ + 20(0.15)(0.85¹⁹) + 190(0.15²)(0.85¹⁸) = 0.0388 + 0.1368 + 0.2293 = 0.405.
P(Type II error) = 1 − 0.405 = 0.595. The test misses a fall from 0.3 to 0.15 in nearly six cases out of ten, so a non-significant result here is weak evidence for H₀ rather than support for it.
M1 for comparing the tail probabilities with 0.05, A1 for X ≤ 2, B1 for the size 0.0355, M1 for the binomial terms at p = 0.15, A1 for 0.405, A1 for 0.595, B1 for the closing comment.

Question 13

A test of H₀: λ = 5 against H₁: λ > 5 uses a single observation X ~ Po(λ) with critical region X ≥ 10. Write down the power function of the test as a function of λ. Given that P(X ≤ 9) = 0.2424 when λ = 12, find the power at λ = 12. Explain why the power function is an increasing function of λ, and state the value of the power function at λ = 5, explaining what that value represents.

Accepted Answer

Power(λ) = P(X ≥ 10 | λ) = 1 − P(X ≤ 9 | λ) = 1 − e^−λ(1 + λ + λ²/2! + … + λ⁹/9!).
At λ = 12: 1 − 0.2424 = 0.7576.
Raising λ shifts the whole Poisson distribution towards larger values, so the probability of a count of 10 or more can only rise: the power increases with λ.
At λ = 5 the power function takes the value P(X ≥ 10 | λ = 5) = 1 − 0.9682 = 0.0318, which is the size of the test: the power function evaluated at the null value is the probability of a Type I error. B1 for the power function, M1 A1 for 0.7576, B1 for the reason the function increases, B1 for 0.0318 identified as the size.

Question 14

X ~ B(20, p) is used to test H₀: p = 0.3 against H₁: p < 0.3 with critical region X ≤ 2. Show that the power function can be written as (1 − p)¹⁸(1 + 18p + 171p²), and use this form to find the power at p = 0.1. State the values of the power function at p = 0 and at p = 1, and explain what each means for the test.

Accepted Answer

Power(p) = P(X ≤ 2) = (1 − p)²⁰ + 20p(1 − p)¹⁹ + 190p²(1 − p)¹⁸. Every term contains (1 − p)¹⁸, so take it out: (1 − p)¹⁸[(1 − p)² + 20p(1 − p) + 190p²]. The bracket is 1 − 2p + p² + 20p − 20p² + 190p² = 1 + 18p + 171p², giving (1 − p)¹⁸(1 + 18p + 171p²).
At p = 0.1: 0.9¹⁸ × (1 + 1.8 + 1.71) = 0.1501 × 4.51 = 0.677.
At p = 0 the value is 1: if successes are impossible then X = 0 in every sample and the test always rejects H₀, which is the correct decision. At p = 1 the value is 0: every trial succeeds, X = 20 always, and the test never rejects; since p = 1 is not in H₁ this is harmless, and it shows the region X ≤ 2 is pointed in the right direction. M1 for the three binomial terms, M1 for taking out (1 − p)¹⁸, A1 for the quadratic 1 + 18p + 171p², A1 for 0.677, B1 for the value 1 at p = 0 with its meaning, B1 for the value 0 at p = 1 with its meaning. Between these the function falls from 1 to the size 0.0355 at p = 0.3 and on towards 0.

Question 15

X ~ B(20, p) is used to test H₀: p = 0.3 against H₁: p < 0.3. The critical region X ≤ 2 has size 0.0355 and power 0.4049 at p = 0.15. Given that P(X ≤ 3 | p = 0.3) = 0.1071, find the power of the wider region X ≤ 3 at p = 0.15. A quality manager says that failing to detect a fall to p = 0.15 is far more serious than a false alarm. Decide which region the manager should use, and state which region must be used if the test is required to have size at most 5%.

Accepted Answer

Power of X ≤ 3 at p = 0.15 = P(X ≤ 3 | p = 0.15) = 0.85²⁰ + 20(0.15)(0.85¹⁹) + 190(0.15²)(0.85¹⁸) + 1140(0.15³)(0.85¹⁷) = 0.0388 + 0.1368 + 0.2293 + 0.2428 = 0.6477.
Widening the region raises the power at p = 0.15 from 0.405 to 0.648, so the chance of the serious error, missing the fall, drops from 0.595 to 0.352. The size rises from 0.0355 to 0.1071, so false alarms become three times as likely. Since the manager rates the missed fall as far more serious, the wider region X ≤ 3 is the one to use.
If the size must not exceed 5%, then 0.1071 rules the wider region out and X ≤ 2 must be used, accepting the lower power. M1 for the four binomial terms at p = 0.15, A1 for 0.6477, B1 for the gain in power against the rise in size, B1 for choosing X ≤ 3 with the reason, B1 for X ≤ 2 under a 5% constraint.