All lessons
Statistics for analysts: from averages to A/B tests
The statistics data and BI analysts use every week: describing data, probability, distributions, sampling, confidence intervals, hypothesis tests and A/B testing, with simulations you can play with and every formula worked by hand.
What you will be able to do
- Describe any dataset honestly: pick the right average, measure spread, spot skew and outliers, and say where one value stands.
- Reason with chance: conditional probability, Bayes, expected value and the distributions behind counts, waiting times and money.
- Go from a sample to the truth: standard errors, confidence intervals, the bootstrap and how big a sample you need.
- Test ideas: t-tests, chi-square, ANOVA and rank tests, what a p-value means, power, and the traps of many tests at once.
- Run and read A/B tests like a product analyst: design, sample size, peeking, sample ratio mismatch and a written decision.
Every lesson has a picture or simulation, a maths box with a worked example, practice questions and a short quiz. You finish with an experiment readout, a one-page decision memo built from a realistic A/B test, ready for your portfolio. Statistics links to the machine learning course From a straight line to an LLM, which covers fitting models, and to fibub's free statistics tools for your own data.
0 of 50 lessons done
Without the tick, your progress lasts until you close this tab. Either way it stays in this browser: there is no sign-up and nothing is sent to us.
All modules
Part A: Describing data and chance
1 Describing data 5 lessons
2 Probability for decisions 4 lessons
3 Distributions 6 lessons
Part B: From a sample to the truth
4 Samples 5 lessons
5 Estimation 4 lessons
6 Hypothesis tests 9 lessons
- 6.1 The logic of a test: null, alternative and evidence
- 6.2 P-values: what they are and what they are not
- 6.3 t-tests: one sample, two samples and paired
- 6.4 Type I and type II errors, and power
- 6.5 Chi-square tests for categories
- 6.6 ANOVA: comparing three or more groups
- 6.7 When the data is not normal: rank tests
- 6.8 Many tests at once: the multiple comparisons trap
- 6.9 Effect size: significant is not the same as important
Part C: Statistics at work
7 A/B testing 8 lessons
- 7.1 Designing an A/B test
- 7.2 Choosing metrics: primary, secondary and guardrails
- 7.3 Sample size and how long to run
- 7.4 The peeking problem
- 7.5 Reading a conversion test from start to finish
- 7.6 Sample ratio mismatch: when the split is broken
- 7.7 The Bayesian view of an A/B test
- 7.8 Novelty effects, interference and when not to test