Confidence intervals and margin of error

A confidence interval is a range, computed from a sample, that quantifies how much the sample statistic might differ from the population parameter through chance alone. The margin of error is the half-width of that range.

The working formulas, which are all this exam needs:

For a proportion: the standard error is the square root of p(1 - p) / n, where p is the sample proportion. The 95 per cent margin of error is 1.96 times that, and 1.96 is close enough to 2 for mental arithmetic.

For a mean: the standard error is s divided by the square root of n, where s is the sample standard deviation. The 95 per cent margin of error is again 1.96 standard errors.

Note the difference between the standard deviation and the standard error, because the exam tests it directly. The standard deviation describes the spread of individual observations and does not shrink as you collect more data; a population is as variable as it is. The standard error describes the spread of the sample mean across hypothetical repeated samples, and it does shrink, because larger samples produce more stable means. Quoting a standard deviation where a standard error belongs makes an estimate look far less precise than it is; quoting a standard error where a standard deviation belongs makes a population look far more uniform than it is.

Worked example 6: What is actually inside the 41 per cent "Other"

Ashcombe logs 240,000 support tickets a year and 41 per cent of them, which is 98,400 tickets, carry the category "Other". An analyst draws a simple random sample of 400 of those and reads them. 108 turn out to be about scheduling conflicts, where two jobs are booked into the same engineer slot.

The finding to report is: "scheduling conflicts account for an estimated 27 per cent of uncategorised tickets, with a 95 per cent confidence interval of 22.7 to 31.4 per cent, which is between 22,300 and 30,900 tickets a year." The point estimate alone would be a false precision, and the interval alone would be unhelpfully vague. Both, with the population count they translate into, is the professional form.

What the interval means, stated carefully, because the exam tests exactly this wording: if you repeated this sampling procedure many times, about 95 per cent of the intervals produced would contain the true population proportion. It does not mean there is a 95 per cent probability that the true value lies in this particular interval. The true value is a fixed number; it is either in this interval or it is not. The 95 per cent is a property of the method, not of the one result you happen to have. In everyday conversation the distinction rarely changes the decision, and in an exam it is the difference between the right and the wrong option.

Now suppose the head of support wants the estimate to within 2 percentage points, because a decision to build a conflict-detection feature hinges on whether the number is above or below 25 per cent.

Halving the margin of error from 4.35 to about 2.2 points needs roughly four times the sample; getting to 2.0 points needs 1,893. At perhaps three minutes a ticket, that is 75 hours of reading. The right conversation with the head of support is not statistical, it is about whether the decision genuinely turns on 2 points, and it usually does not, because a feature that addresses 22,300 tickets a year is worth building on the same grounds as one addressing 30,900.

One caution on this example. The interval covers sampling error only. It assumes the 400 tickets were drawn at random from the 98,400, and it assumes the analyst classified them consistently. If the sample was "the first 400 in the export", which is ordered by date, then the interval is a precise statement about a biased sample and the real uncertainty is larger and unmeasured. A confidence interval never covers bias. It is the most common misreading of a margin of error in commercial work: treating a narrow interval as evidence that the estimate is close to the truth, when it is only evidence that the estimate is stable under resampling.