BiologyScientific method and quantitative biology › Statistics for biologists: spread, error bars and the three tests

Statistics for biologists: spread, error bars and the three tests

Biological material varies, so a difference between two means is never self-evidently a difference. A statistical test is the tool that asks how easily the variation you already have could have produced the difference you are excited about.

Before this Calculating a mean · Drawing and reading a scatter graph

COMMON MISCONCEPTION

If the error bars on two means do not overlap, the difference between them is significant.

What you should be able to do

Mean, median, and the reason a range is not enough

The first job of a statistic is to stand in for a whole set of readings, and which one you choose depends on what those readings look like.

Mean
The total divided by the number of readings. Uses every value, and is what almost every biological calculation wants.
Median
The middle value when the readings are put in order. Unmoved by a single extreme value, which is why it is used for skewed data such as incomes, reaction times or the number of parasites per host.
Mode
The most common value. The only one of the three that works for categories such as blood group or flower colour.

One number is never enough, because two sets of readings with the same mean can be entirely different sets. What goes alongside it is a measure of spread, and the obvious candidate, the range, turns out to be a poor one.

Same mean, same range, and the two samples are plainly not alike. The range is decided by the two most extreme readings and is blind to everything between them; the standard deviation is moved by every reading in the set.

The range uses two numbers and discards the rest, so it cannot tell readings that cluster tightly with two outliers from readings spread evenly. It also gets worse as you collect more data, since more readings means a better chance of catching an extreme one, and a statistic that depends on how long you spent measuring is not describing the organism. The standard deviation fixes both problems: it is roughly the typical distance of a reading from the mean, and every reading contributes to it.

s = √[ Σ(x − x̄)² ÷ (n − 1) ]sum the squared deviations from the mean, divide by one less than the number of readings, take the square root

The n − 1 is not a typo. The sample's own mean is being used to measure the sample's spread, which makes the deviations slightly too small, and dividing by n − 1 corrects for it. A version with n on the bottom is also a real formula, and it belongs to a complete population, where the mean is the population's own and there is nothing to correct. That is why a calculator offers two standard deviation keys, usually labelled sample and population, or s and σ. Biological readings are a sample drawn from something larger, so n − 1 is the one to reach for, and it is the form written on the formula sheets that print a standard deviation for biology. If a question gives you a formula, use that one.

A standard deviation from nine readings

Sample A consists of the nine readings 14, 19, 19, 20, 20, 20, 21, 21 and 26. Sample B consists of 14, 15, 17, 19, 20, 21, 23, 25 and 26. Both sets are idealised, chosen so that the arithmetic stays visible. Calculate the mean and standard deviation of each and comment.

Show the working

Both totals are 180, so both means are 180 ÷ 9 = 20; both medians are the fifth value in order, which is 20; both ranges are 26 − 14 = 12. On every statistic so far the samples are identical.

For sample A the deviations from the mean are −6, −1, −1, 0, 0, 0, +1, +1 and +6, whose squares total 76. Then 76 ÷ 8 = 9.5, and √9.5 = 3.08.

For sample B the deviations are −6, −5, −3, −1, 0, +1, +3, +5 and +6, whose squares total 142. Then 142 ÷ 8 = 17.75, and √17.75 = 4.21.

Sample B is about a third more variable than sample A, and nothing except the standard deviation said so. Squaring does the work: it makes every deviation positive so they cannot cancel, and it weights a reading six from the mean thirty-six times as heavily as one a single unit away.

Error bars, and the claim they do not support

An error bar shows how much the readings behind a mean varied. It turns a column that looks like a fact into a column with a width, which is the truthful way to draw a biological measurement.

The middle panel is the only one where the eye is being told anything useful, and even there the test still has to be done. The right-hand panel is where every intuition about error bars comes unstuck.

The first question about any error bar is what it was drawn from, because three things are commonly used and they mean different things:

BarWhat it showsWhat happens as the sample grows
± 1 standard deviationHow much the individual readings varyStays about the same: it is a property of the organism
± 1 standard errorHow precisely the mean has been estimatedShrinks, roughly as one over the square root of n
95% confidence intervalA range that would contain the true mean in 95 of 100 such samplesShrinks, and is roughly twice the standard error for a decent sample

That table is why the overlap rule fails. Standard-deviation bars describe variation between organisms and barely narrow however many you measure, so two genuinely different means can easily have overlapping SD bars. Standard-error bars describe how well you know the mean and shrink with sample size, so with a large enough sample almost any two means separate. The same data drawn two ways gives opposite impressions, and neither drawing is a test.

What the bars are genuinely good for is triage. Large overlap means a significant difference is unlikely and the test will probably say so; no overlap means there may well be one and the test is worth doing. Everything between belongs to the test alone, and the sentence to write is 'the error bars do not overlap, which suggests a difference; a t-test is needed to establish whether it is significant.'

Error bars also earn a mark by saying what varied. If the bars on a graph of enzyme activity are long at 50 °C and short at 20 °C, that is a finding: the reaction has become less repeatable near the denaturation threshold, and saying so is better biology than describing the means alone.

The null hypothesis, and what p = 0.05 actually says

Every statistical test starts from a null hypothesis: a specific statement that there is no effect. It is written before the data are looked at, and the test's one job is to ask whether the data are awkward for it.

Null hypothesis
A statement that there is no significant difference, no significant association or no significant correlation between the variables, and that any difference observed is due to chance.
Significance level
The probability threshold, conventionally 0.05, below which the null hypothesis is rejected.
Degrees of freedom
A number derived from the size of the data set that decides which critical value you read from the table.

Writing it well is a mark on its own, and the failure mode is vagueness. Name both variables and the populations:

TestA null hypothesis in the form examiners accept
Chi-squaredThere is no significant difference between the observed numbers of offspring in each phenotype and the numbers expected from a 9:3:3:1 ratio.
t-testThere is no significant difference between the mean shell length of limpets on the sheltered shore and the mean shell length of limpets on the exposed shore.
Spearman's rankThere is no significant correlation between soil moisture content and the number of plant species per quadrat.

Now the part worth getting right rather than merely learning. A p-value is the probability of getting a result at least as extreme as the one you got, if the null hypothesis were true. Textbooks shorten that to 'the probability that the difference is due to chance' and mark schemes accept it, so you may write it. Keep the long version in your head, though, because the short one invites two conclusions that are false.

It does not say there is a 95% probability the effect is real. The p-value is calculated assuming the null hypothesis is true, so it cannot then say how likely that assumption was. And it says nothing about the size of the effect: half a millimetre between two mean shell lengths will come out significant if you measure enough limpets, because significance depends on sample size as much as on the difference. Significant means unlikely to be this big by chance alone, and the sentence after a significant result should always say how big the difference was as well as that it was there.

The 0.05 is a convention and it sets what you are willing to risk: one time in twenty you will reject a true null hypothesis and announce an effect that is not there. That is the cost of the test, and it is why a study that runs twenty comparisons and reports the one that came out significant has reported nothing.

Choosing the test

Three tests cover almost everything an A-level investigation produces, and choosing between them is a question about the shape of your data rather than about the biology.

Work from the data you collected, not from the topic. Counts in categories, two separate means, or two quantities asked whether they rise and fall together: those three questions separate the tests.
You haveYou want to knowTest
Counts of individuals in categories, and a ratio you predictedWhether the counts differ from the prediction more than chance would explainChi-squared
Two sets of measurements, each scattered about a meanWhether the two means differStudent's t-test
Two different quantities measured on each of the same individuals or quadratsWhether they rise and fall togetherSpearman's rank correlation

Two conditions are worth checking first. Chi-squared needs raw counts, never percentages, and every expected value should be at least 5; combining categories or collecting more data is the fix when they are not. The t-test assumes measurements that are continuous and roughly normally distributed, which lengths, masses and volumes usually are and counts of individuals usually are not, and it assumes the two samples are about equally spread. That last condition is the one nobody mentions, and the next section explains why the A-level formula depends on it.

One distinction saves a common error: 'two measurements per individual' does not automatically mean Spearman. Spearman asks whether two different quantities are associated, moisture against species count. Measuring the same quantity twice on each individual, before and after a treatment, is a difference question, and the tool built for it is the paired t-test, a variant A-level papers rarely set but which exists; if a question supplies before-and-after data, work with the differences rather than reaching for a correlation.

All three then work the same way. Calculate a number from your data, work out the degrees of freedom, look up the critical value at p = 0.05, and compare. For all three, a calculated value at or above the critical value means reject the null hypothesis. Spearman's is the one to watch, because its value runs from −1 to +1 and it is the size you compare: a correlation of −0.87 is exactly as significant as one of +0.87 and points the other way.

The three tests, worked

The numbers below are constructed rather than collected, chosen so the arithmetic stays readable. The sequence of steps is identical every time.

chi-squared = Σ [ (O − E)² ÷ E ]O is the observed count, E the expected count, and the sum runs over every category

Chi-squared against a predicted ratio

A dihybrid cross is predicted to give offspring in the ratio 9:3:3:1. Of 320 offspring, the four phenotypes are counted as 168, 66, 70 and 16. Test whether the results differ significantly from the prediction.

Show the working

Null hypothesis: there is no significant difference between the observed numbers of each phenotype and the numbers expected from a 9:3:3:1 ratio.

Expected values. The ratio has 9 + 3 + 3 + 1 = 16 parts and 320 ÷ 16 = 20 offspring per part, so the expected counts are 180, 60, 60 and 20, every one well above 5.

The sum. (168 − 180)² ÷ 180 = 0.80; (66 − 60)² ÷ 60 = 0.60; (70 − 60)² ÷ 60 = 1.67; and (16 − 20)² ÷ 20 = 0.80. Adding them gives chi-squared = 3.87.

Degrees of freedom = number of categories − 1 = 4 − 1 = 3. The critical value at p = 0.05 with 3 degrees of freedom is 7.81.

Conclusion. 3.87 is less than 7.81, so the null hypothesis is not rejected. There is no significant difference between the observed results and a 9:3:3:1 ratio, and the differences that exist can be attributed to chance, a result consistent with two genes assorting independently.

The t-test needs a sentence about which t-test, because more than one exists and A-level teaches one of them. The version below is the one Cambridge International 9700 supplies to candidates, and it is the version an A-level biology question will normally print for you. It comes with a condition that is easy to miss: the two samples should have standard deviations of roughly the same size. Where that holds, the arithmetic below and the n₁ + n₂ − 2 degrees of freedom go together and the result is sound.

t = (x̄₁ − x̄₂) ÷ √( s₁²/n₁ + s₂²/n₂ )take the difference between the means as a positive number; degrees of freedom = n₁ + n₂ − 2, on samples whose standard deviations are of similar size

Where it does not hold, statistics beyond A-level splits into two named tests, and it is worth knowing they are there so that a textbook or a spreadsheet does not confuse you. Welch's t-test uses exactly the denominator above but calculates its degrees of freedom from the two variances rather than from n₁ + n₂ − 2. The pooled Student t-test, which is the one n₁ + n₂ − 2 properly belongs to, combines the two spreads into a single pooled estimate first. The A-level formula borrows the denominator from one and the degrees of freedom from the other, which is a deliberate simplification and works because the samples in an A-level question are chosen to have similar spreads. Two rules follow. Use the formula your question or your board's sheet supplies, degrees of freedom included, because that is the method being marked. And if the two standard deviations are wildly different, say so in the evaluation, because a limitation you can name is a limitation worth a mark.

A t-test on two means

Limpet shell lengths are measured on a sheltered shore and on an exposed shore, ten from each. The sheltered sample has a mean of 22.0 mm with a standard deviation of 2.0 mm; the exposed sample has a mean of 19.0 mm with a standard deviation of 2.5 mm. Test whether the means differ significantly.

Show the working

Null hypothesis: there is no significant difference between the mean shell length of limpets on the sheltered shore and the mean shell length of limpets on the exposed shore.

Check the condition. The two standard deviations are 2.0 mm and 2.5 mm, close enough in size for the formula and its n₁ + n₂ − 2 degrees of freedom to belong together. Had one sample been five times as variable as the other, the test would still be the one to run and the evaluation would have to say that the result is approximate.

The denominator first. s₁²/n₁ = 4.0 ÷ 10 = 0.400 and s₂²/n₂ = 6.25 ÷ 10 = 0.625, totalling 1.025, and √1.025 = 1.012.

The statistic. The means differ by 22.0 − 19.0 = 3.0 mm, so t = 3.0 ÷ 1.012 = 2.96.

Degrees of freedom = 10 + 10 − 2 = 18, and the critical value at p = 0.05 is 2.10.

Conclusion. 2.96 is greater than 2.10, so the null hypothesis is rejected. There is a significant difference between the means: limpets on the sheltered shore have shells 3.0 mm longer on average, and a difference this large would arise in fewer than 5 samples in 100 if the two shores really held the same population.

rs = 1 − [ 6 Σd² ÷ n(n² − 1) ]d is the difference between the two ranks of one individual, and n is the number of individuals, not the number of measurements

Spearman's rank on ten quadrats

Soil moisture and the number of plant species are recorded in each of ten quadrats. Ranking each variable from 1 to 10 and taking the difference in rank for each quadrat gives squared differences of 1, 4, 4, 1, 1, 4, 4, 1, 1 and 1. Test for a correlation.

Show the working

The measurements are not reproduced here on purpose. Spearman's test never sees them, because it replaces every value with its position in the order, so the ranks are the whole of the input and printing invented moisture readings would only invite you to interpret scatter nobody measured.

Null hypothesis: there is no significant correlation between soil moisture content and the number of plant species per quadrat.

The statistic. Σd² = 22, and with n = 10, n(n² − 1) = 10 × 99 = 990. So rs = 1 − (6 × 22) ÷ 990 = 1 − 0.133 = 0.87.

The critical value for n = 10 at p = 0.05 is 0.648. Spearman uses n, the number of pairs, rather than a degrees-of-freedom figure.

Conclusion. 0.87 is greater than 0.648, so the null hypothesis is rejected: there is a significant positive correlation between soil moisture and the number of plant species. It is positive because rs is positive, and being significant does not make it causal, because both could be responding to something else about where the quadrats fell.

Ranking has one detail that gets tested: equal readings take the mean of the positions they occupy. Values of 3, 7, 7 and 9 rank as 1, 2.5, 2.5 and 4, because the two sevens would have held positions 2 and 3, and the next value still takes position 4.

Writing the conclusion

Every conclusion from these tests has the same four parts, and mark schemes award them separately. Leave one out and you lose it, however good the arithmetic was.

The partWhat it looks like
The comparison'The calculated value of 2.96 is greater than the critical value of 2.10…'
The conditions'…at p = 0.05 with 18 degrees of freedom.'
The decision'The null hypothesis is therefore rejected.'
The biology'There is a significant difference between the mean shell lengths on the two shores.'

Two points of wording. When the calculated value falls short, the sentence to write is that the null hypothesis is not rejected, or that the data fail to reject it. Plenty of biology textbooks and mark schemes print 'the null hypothesis is accepted' and will credit it, so writing that costs nothing, but the longer form is the one that says what actually happened: the data gave no reason to abandon the null hypothesis, which is a long way from showing it true, because a small sample fails to reject almost anything. Pearson's guidance for the maths qualifications now asks explicitly for non-assertive wording, and the habit is worth having in both subjects. If there is room for one extra clause in an evaluation, 'a larger sample might have detected a difference this one could not' is the clause worth having.

And keep 'significant' for what the test said. In ordinary English it means important; here it means the calculated value beat the critical value at the stated probability. Calling a difference significant when no test was done, and calling a significant difference large, are two ways of losing the same mark.

TRY IT: Reading a result properly

A student compares the mean number of stomata per square millimetre on the upper and lower surfaces of privet leaves, twelve leaves of each. The calculated value of t is 1.84 and the critical value at p = 0.05 with 22 degrees of freedom is 2.07. Write the conclusion, and say what the student should do next.

Check your answer

The calculated value of 1.84 is less than the critical value of 2.07 at p = 0.05 with 22 degrees of freedom, so the null hypothesis is not rejected. There is no significant difference between the mean number of stomata per square millimetre on the two surfaces of the leaves sampled.

The student must not write that the two surfaces have the same stomatal density. Failing to detect a difference is not the same as showing there is none, and twelve leaves per surface gives the test limited power to detect a small one.

What to do next is plan a larger study, its size decided in advance: a larger sample raises the degrees of freedom, lowers the critical value and shrinks the standard error of each mean. Plan it as a fresh investigation rather than topping up this data set until the answer changes, because stopping when a result appears inflates the chance of a false positive. The sampling is worth checking too. Twelve leaves from one plant describe that plant rather than privet, and no statistics repair a sample drawn badly.

In the exam

Check yourself

Two groups of woodlice are given a choice chamber, one half humid and one half dry. Of 80 woodlice, 58 are found on the humid side and 22 on the dry side after ten minutes. State a null hypothesis, choose and carry out a suitable test, and write the conclusion. The critical value of chi-squared at p = 0.05 with 1 degree of freedom is 3.84.

Answer

Test: chi-squared, because the data are counts of individuals in two categories against a ratio that can be predicted, since with no effect of humidity the woodlice would be spread 1:1.

Null hypothesis: there is no significant difference between the observed numbers of woodlice on the humid and dry sides and the numbers expected if humidity had no effect on their distribution.

Calculation: the expected values are 80 ÷ 2 = 40 on each side, both well above 5. Then (58 − 40)² ÷ 40 = 8.10 and (22 − 40)² ÷ 40 = 8.10, so chi-squared = 16.2. Degrees of freedom = 2 categories − 1 = 1, and the critical value is 3.84.

Conclusion: 16.2 is far greater than 3.84 at p = 0.05 with 1 degree of freedom, so the null hypothesis is rejected. Significantly more woodlice were found on the humid side than an even distribution would give.

Worth adding in an evaluation: the test shows the distribution was uneven, not why. Separating a kinesis, in which woodlice simply move less once they reach damp air and so accumulate, from a taxis, in which they move towards it directionally, needs individual paths tracked rather than a headcount at the end.

Questions

Written to the command words the boards use. Try them on paper before opening a scheme: the marks go to points made, not to length.

Question 15 marks

Two heterozygous pea plants are crossed and the offspring are expected in a ratio of 3 tall to 1 dwarf. Of 400 offspring, 280 are tall and 120 are dwarf. Calculate chi-squared and use it to reach a conclusion. The critical value at p = 0.05 with 1 degree of freedom is 3.84.

Mark scheme
  1. M1 expected values are 400 × 3/4 = 300 tall and 400 × 1/4 = 100 dwarf, both comfortably above 5
  2. M1 (280 − 300)² ÷ 300 = 1.33 and (120 − 100)² ÷ 100 = 4.00
  3. A1 chi-squared = 1.33 + 4.00 = 5.33
  4. B1 degrees of freedom = 2 categories − 1 = 1, and the calculated value of 5.33 is greater than the critical value of 3.84 at p = 0.05
  5. B1 the null hypothesis is rejected: there is a significant difference between the observed numbers and a 3:1 ratio, so the difference is unlikely to have arisen by chance alone

Question 24 marks

Five leaves taken from one shrub have widths of 12, 14, 15, 17 and 22 mm. Calculate the mean width and the standard deviation of these widths, giving the standard deviation to three significant figures.

Mark scheme
  1. M1 mean = (12 + 14 + 15 + 17 + 22) ÷ 5 = 80 ÷ 5 = 16 mm
  2. M1 the deviations from the mean are −4, −2, −1, +1 and +6, and the squares of these total 16 + 4 + 1 + 1 + 36 = 58
  3. M1 divide by n − 1 rather than by n: 58 ÷ 4 = 14.5, then take the square root
  4. A1 standard deviation = 3.81 mm

Question 34 marks

A student writes that the error bars on two mean values do not overlap, so the difference between the means is significant. Explain why this reasoning is unsound.

Mark scheme
  1. B1 an error bar may show one standard deviation, one standard error or a 95 per cent confidence interval, and the three have different lengths drawn on the same data, so the same means can appear to overlap or not depending on the choice
  2. B1 standard-deviation bars describe variation between individuals and barely narrow however many organisms are measured, so two genuinely different means can easily have overlapping bars
  3. B1 standard-error bars describe how precisely each mean has been estimated and shrink roughly as one over the square root of the sample size, so with a large enough sample almost any two means separate
  4. B1 significance is decided only by comparing a calculated test value with a critical value at a stated probability, so bars can suggest that a test is worth doing and no more

Question 44 marks

Compare Student's t-test with Spearman's rank correlation, referring to the data each one needs and the question each one answers.

Mark scheme
  1. B1 a t-test needs two separate sets of measurements each scattered about a mean, whereas Spearman's needs two different quantities measured on each of the same individuals or quadrats
  2. B1 a t-test asks whether the two means differ, whereas Spearman's asks whether the two variables vary together
  3. B1 the t-test uses the measurements themselves and assumes they are continuous and roughly normally distributed, whereas Spearman's replaces every measurement by its rank and never sees the values
  4. B1 degrees of freedom for the t-test are n₁ + n₂ − 2, whereas Spearman's critical value is read against n, the number of pairs; for both, a calculated value at or above the critical value means the null hypothesis is rejected

Question 53 marks

A researcher measures 2000 fish from each of two lakes and reports that the mean masses differ by 0.4 g, a difference a t-test finds significant at p = 0.05. Suggest why this significant result may still be of little biological interest.

Mark scheme
  1. B1 significance depends on the size of the sample as well as on the size of the difference, so a sample of 2000 from each lake can detect a difference this small
  2. B1 significant means unlikely to have arisen by chance under the null hypothesis; it does not mean the difference is large, and 0.4 g is a tiny fraction of the mass of a fish
  3. B1 the sentence reporting a significant result should give the size of the difference as well, and further work would be needed to show whether a difference this small has any effect on the fish

Question 62 marks

An investigation compares the mean number of stomata per square millimetre on leaves grown in sun and on leaves grown in shade. State a suitable null hypothesis, and state the significance level conventionally used in biology.

Mark scheme
  1. B1 there is no significant difference between the mean number of stomata per square millimetre on sun leaves and the mean number on shade leaves
  2. B1 the conventional significance level is p = 0.05, below which the null hypothesis is rejected

Worth remembering

  • The mean uses every reading, the median ignores extremes, the mode is the only one that works on categories.
  • The range is fixed by two readings and grows with sample size; the standard deviation uses them all.
  • Error bars can suggest a difference and can never establish one, and what they suggest depends on whether they show SD, standard error or a confidence interval.
  • The null hypothesis says there is no significant difference, association or correlation, and is written before the data are seen.
  • Counts against a predicted ratio go to chi-squared, two means to a t-test, two measurements per individual to Spearman's rank.
  • The A-level t-test formula assumes two samples of roughly equal spread, and its n₁ + n₂ − 2 degrees of freedom belong to that assumption.
  • Calculated value at or above the critical value means reject the null hypothesis; below it means the data give no reason to.
  • Significant means unlikely to have arisen by chance under the null hypothesis. It does not mean large, and it does not mean proved.

CHECK YOUR PROGRESS

Rate how confident you are with each objective for this lesson. Ratings are kept in this browser, on this device, and are sent nowhere.

  • Choose between a mean, a median and a mode for a given set of biological data, and say why.
  • Calculate a standard deviation and explain what it shows that a range does not.
  • Interpret error bars, including what non-overlapping bars do and do not prove.
  • State a null hypothesis in the form a mark scheme accepts.
  • Choose between chi-squared, a t-test and Spearman's rank for a given question, and find the degrees of freedom.
  • Compare a calculated value with a critical value and write the conclusion in the wording examiners want.

Open the full revision checklist to see every objective in the curriculum in one place.

WORKBOOK

The same questions as the player, on paper with room to work, and a separate book of mark schemes. Free to use; please do not redistribute or sell.