All lessons

Module 4: Samples · Lesson 4.4 · about 20 min

Standard error: how much a sample result wobbles

The standard error measures how far a statistic typically lands from the truth: σ/√n for a mean, √(p(1−p)/n) for a proportion, and the formula for a difference between two groups. SD versus SE, and the square-root law.

Key points, meanings and formulas on one page: choose "Save as PDF" in the print box.

Module 4 · about 20 minutes · school maths. You’ll be able to compute standard errors for a mean, a proportion and a difference, tell SE from SD, and plan with the square-root law.

The standard error is the typical distance between a sample result and the truth: the size of the wobble you would see if you repeated the study.

In everyday terms: measure your own height ten times with a tape and you get slightly different numbers. The SD is how much single measurements scatter; the SE is how much the average of your ten measurements would scatter if you did the whole thing again.

1. The size of the wobble

The standard error (SE) is the standard deviation of a statistic’s sampling distribution: how far, typically, the estimate lands from the truth. It is the number behind every margin of error, confidence interval and test in the rest of the course.

Try it: how fast does the wobble shrink?
standard error σ/√n
±2 SE band
n needed to halve it
Basket values have mean $40 and SD $20 (made up). The shaded band is the mean ± 2 standard errors: about 95 % of sample means land inside it. The band narrows fast at first and then slowly, because it shrinks with the square root of n. Going from 25 to 100 baskets halves it; halving it again takes 400.

2. SD versus SE

Standard deviation Standard error
Describes Individual values An estimate (a mean, a rate)
Question How different are customers from each other? How sure are we about the average?
With more data Settles near σ Shrinks towards 0

A common reporting error is to put ”± SE” on a chart where readers expect ”± SD”, or the other way round. Say which one you’re showing.

3. The formulas

Mean:

SE(x‾)=sn

using the sample SD s in place of the unknown σ.

Proportion:

SE(pˆ)=pˆ(1−pˆ)n

Difference between two independent groups (A/B tests!): variances add, so

SE(x‾B−x‾A)=sA2nA+sB2nBSE(pˆB−pˆA)=pˆA(1−pˆA)nA+pˆB(1−pˆB)nB

A difference is noisier than either group alone, about √2 times as noisy for equal groups. This is why A/B tests need more traffic than people expect.

4. The square-root law

Because of the n, precision gets expensive: 4× the data halves the SE; 100× the data cuts it to a tenth. When planning, ask what precision you need before asking for more data. Going from a ±2-point to a ±1-point margin costs four times the sample.

SD
Spread of individual values (customers, orders).
SE of a mean = s/√n
Spread of the sample mean across repeated samples.
SE of a share = √(p(1 − p)/n)
Spread of a percentage such as a conversion rate.
SE of a difference = √(SE₁² + SE₂²)
Wobbles add in squares when you compare two groups.
Physical meaning
The SE is the "give or take" on every result you report. To halve it you need four times as much data, which is why precision gets expensive.

“The SE of a difference is SE₁ + SE₂.” Two independent wobbles partly cancel, so they add in squares: √(0.28² + 0.30²) ≈ 0.41 points, not 0.58.

Watch it happen, one step at a time:

The test. A: 200 of 5,000 bought (4.0 %). B: 240 of 5,000 (4.8 %).
SE of A. √(0.04 × 0.96 ÷ 5,000) ≈ 0.28 points.
SE of B. √(0.048 × 0.952 ÷ 5,000) ≈ 0.30 points.
SE of the difference. √(0.28² + 0.30²) ≈ 0.41 points.
Compare. The difference is 0.8 points, about 2 SEs: borderline, worth the formal test of Module 6.
Maths and principle

Principle: the SE of an average equals the spread of one value divided by the square root of how many values were averaged.

Equations: SE(x‾)=s/n; SE(pˆ)=pˆ(1−pˆ)/n; for independent groups SE(diff)=SEA2+SEB2.

In code: Excel =STDEV.S(range)/SQRT(COUNT(range)); pandas df["x"].sem(); SQL STDDEV_SAMP(x) / SQRT(COUNT(x)).

5. Worked example

Made-up numbers. A test of a new product page:

Visitors Buyers Rate
A (old) 5,000 200 4.0 %
B (new) 5,000 240 4.8 %
  • SE(A) = √(0.04 × 0.96 ÷ 5,000) ≈ 0.00277 (0.28 points).
  • SE(B) = √(0.048 × 0.952 ÷ 5,000) ≈ 0.00302 (0.30 points).
  • SE(difference) = √(0.00277² + 0.00302²) ≈ 0.0041 (0.41 points).
  • Observed difference 0.8 points is about 0.8 ÷ 0.41 ≈ 1.95 SEs. Borderline: right at the usual line for “clearly more than noise”. Module 7 takes this further.

6. Practice

  1. 25 values with s = 10. What is the SE of the mean?
  2. A rate of 20 % from 1,600 people. What is the SE?
  3. Group A: mean 50, s = 12, n = 36. Group B: mean 55, s = 16, n = 64. What is the SE of the difference?
Answers
  1. 10 ÷ 5 = 2.
  2. √(0.2 × 0.8 ÷ 1,600) = √0.0001 = 0.01, one percentage point.
  3. √(144/36 + 256/64) = √(4 + 4) = √8 ≈ 2.83. The difference of 5 is about 1.8 SEs.

Try it on your own data: the hypothesis tests tool shows the standard error behind each test it runs; the sample size calculator runs the square-root law backwards.

New words: standard error, SE of a proportion, SE of a difference, square-root law.

  • SD describes individuals; SE describes a statistic.
  • SE of a mean = s/√n; of a share = √(p(1 − p)/n).
  • For a difference of independent groups, add SEs in squares.
  • Square-root law: four times the data halves the SE.

Check yourself

1What is the difference between the standard deviation and the standard error?
2A survey of 400 gives p̂ = 0.5. What is the standard error?
3To cut a standard error by a factor of 3, how much more data do you need?

Sources

Your ticks stay in this browser. Save them as a file on the course home.

Updated 2026-11-22. Numbers in the examples are made up to show the idea unless a source is given. Simulations run in your browser; nothing you type is sent anywhere.