Averages, spread and the null hypothesis

Mean, median, and the reason a range is not enough

The first job of a statistic is to stand in for a whole set of readings, and which one you choose depends on what those readings look like.

Mean

The total divided by the number of readings. Uses every value, and is what almost every biological calculation wants.

Median

The middle value when the readings are put in order. Unmoved by a single extreme value, which is why it is used for skewed data such as incomes, reaction times or the number of parasites per host.

Mode

The most common value. The only one of the three that works for categories such as blood group or flower colour.

One number is never enough, because two sets of readings with the same mean can be entirely different sets. What goes alongside it is a measure of spread, and the obvious candidate, the range, turns out to be a poor one.

Describe the drawing

Two dot plots one above the other on identical horizontal scales. The upper set piles most of its dots on top of one another near the middle with one dot far out on each side; the lower set spreads its dots evenly across the same width. Below each plot a short amber bar marks one standard deviation either side of the mean and a longer coral bar marks the full range, and the two coral bars are the same length while the amber ones are not.

The range uses two numbers and discards the rest, so it cannot tell readings that cluster tightly with two outliers from readings spread evenly. It also gets worse as you collect more data, since more readings means a better chance of catching an extreme one, and a statistic that depends on how long you spent measuring is not describing the organism. The standard deviation fixes both problems: it is roughly the typical distance of a reading from the mean, and every reading contributes to it.

s = √[ Σ(x − x̄)² ÷ (n − 1) ]sum the squared deviations from the mean, divide by one less than the number of readings, take the square root

The n − 1 is not a typo. The sample's own mean is being used to measure the sample's spread, which makes the deviations slightly too small, and dividing by n − 1 corrects for it. A version with n on the bottom is also a real formula, and it belongs to a complete population, where the mean is the population's own and there is nothing to correct. That is why a calculator offers two standard deviation keys, usually labelled sample and population, or s and σ. Biological readings are a sample drawn from something larger, so n − 1 is the one to reach for, and it is the form written on the formula sheets that print a standard deviation for biology. If a question gives you a formula, use that one.

Error bars, and the claim they do not support

An error bar shows how much the readings behind a mean varied, or how precisely the mean is known. It turns a column that looks like a fact into a column with a width, which is the truthful way to draw a biological measurement.

The first question about any error bar is what it was drawn from, because three things are commonly used and they mean different things:

For sample A in the example above, s = 3.08 and n = 9, so SE = 3.08 ÷ √9 = 1.03, and the 95% confidence interval is 20 ± 2.1, from 17.9 to 22.1.

The table is why the overlap rule fails. Standard-deviation bars describe variation between organisms and barely narrow however many you measure, so two genuinely different means can easily have overlapping SD bars. Standard-error bars describe how well you know the mean and shrink with sample size, so with a large enough sample almost any two means separate. The same data drawn two ways gives opposite impressions, and neither drawing is a test.

Bar overlap alone does not establish statistical significance, so the sentence to write is 'the error bars do not overlap, which suggests a difference; a t-test is needed to establish whether it is significant.'

Error bars also describe what varied. If the bars on a graph of enzyme activity are long at 50 °C and short at 20 °C, that is a finding: the reaction has become less repeatable near the denaturation threshold, and saying so is better biology than describing the means alone.

The null hypothesis, and what p = 0.05 actually says

Every statistical test starts from a null hypothesis: a specific statement that there is no effect. It is written before the data are looked at, and the test's one job is to ask whether the data are awkward for it.

Null hypothesis

A statement that there is no significant difference, no significant association or no significant correlation between the variables, and that any difference observed is due to chance.

Significance level

The probability threshold, conventionally 0.05, below which the null hypothesis is rejected.

Degrees of freedom

A number, worked out from the number of categories for chi-squared or from the sample sizes for a t-test, that decides which critical value you read from the table.

Name both variables and the populations in the null hypothesis. The form of the sentence is the same whichever of the three tests below follows it. Next topic: choosing and concluding a statistical test.

A p-value is the probability of obtaining a result at least as extreme as the one observed, if the null hypothesis were true. It is often shortened to 'the probability that the difference is due to chance', which is accepted at A level, but the full statement matters because the short form supports two false conclusions.

It does not say there is a 95% probability the effect is real. The p-value is calculated assuming the null hypothesis is true, so it cannot then say how likely that assumption was. And it says nothing about the size of the effect: half a millimetre between two mean shell lengths will come out significant if you measure enough limpets, because significance depends on sample size as much as on the difference. Significant means unlikely to be this big by chance alone, and the sentence after a significant result should always say how big the difference was as well as that it was there.

The 0.05 is a convention and it sets what you are willing to risk: one time in twenty you will reject a true null hypothesis and announce an effect that is not there. That is the cost of the test, and it is why a study that runs twenty comparisons and reports the one that came out significant has reported nothing.

Describe the drawing

Two drawings of one set of twenty comparisons. Along the top, the twenty p-values are plotted as dots on an axis running from zero to one, with the region below nought point nought five shaded and marked by an upright amber line; one dot falls inside it and is drawn coral while the other nineteen are scattered across the rest of the axis in cyan. Below on the left, the same twenty comparisons are drawn the way a result is reported, each as the mean of group A joined by a line to the mean of group B; nineteen are faint grey and the one that came out below the threshold is picked out in coral, a little steeper than the rest but plainly one of the same set. To the right the arithmetic is set out, with the measured share of tests and of families of twenty that produced a significant result from a population in which there was nothing to find.