Module 6: Hypothesis tests · Lesson 6.5 · about 25 min
Chi-square tests for categories
Tests for counts in categories: goodness of fit (does the mix match what we expect?) and independence (are two categorical columns related?). Expected counts, the χ² sum, degrees of freedom, the small-count rule and Cramér's V.
Module 6 · about 25 minutes · school maths. You’ll be able to run chi-square goodness-of-fit and independence tests, check their small-count rule, and report the strength of the relationship.
In everyday terms: a bag of sweets is meant to be one third each of red, green and yellow. You count yours. Chi-square measures how far your counts are from a fair third each, allowing for the fact that a small bag never splits perfectly.
1. Tests for counts
Many business questions are about categories, not averages:
- Is churn related to the plan a customer is on?
- Do complaints split across regions the way sales do?
- Is the 50/50 split in our A/B test really 50/50? (Lesson 7.6)
The data is a table of counts. The chi-square (χ²) test compares observed counts with the counts you would expect under the null hypothesis:
Each cell adds its squared gap, scaled by how big a count was expected (a gap of 10 matters more when 20 were expected than when 2,000 were).
2. Goodness of fit
One categorical column, compared with expected shares. Degrees of freedom = categories − 1.
Example: complaints by region were 50, 30, 20 (n = 100), and sales shares are 40 %, 40 %, 20%. Expected counts: 40, 40, 20. χ² = 100/40 + 100/40 + 0 = 5.0 with 2 df: p ≈ 0.082. The complaint mix doesn’t differ clearly from the sales mix.
3. Independence (contingency tables)
Two categorical columns cross-tabulated. Under independence, each cell’s expected count is
and df = (rows − 1) × (columns − 1).
| Basic | Plus | Pro | |
|---|---|---|---|
| Renewed | |||
| Left |
Edit the counts and watch the expected (outlined) bars and the χ² change. Make the Pro column’s renewals match the others and the p-value climbs.
4. Strength and the small-count rule
Cramér’s V turns χ² into a 0-to-1 strength measure: , where is the smaller of rows and columns. Rough guide for 2-column tables: 0.1 small, 0.3 medium, 0.5 large. With big data a tiny V can be “significant”; report V too.
Small counts: the χ² approximation needs expected counts of at least about 5 in every cell. If not, combine sparse categories or use Fisher’s exact test (for 2 × 2 tables).
A significant result says the columns are related, not how. Look at which cells have the largest contributions, or at row percentages, to describe it.
- Observed O
- The counts in each cell of your table.
- Expected E
- Row total × column total ÷ grand total: the count if there were no link.
- χ² = Σ (O − E)² ÷ E
- The total squared surprise, scaled by what was expected.
- df = (rows − 1)(cols − 1)
- How many cells are free to vary once totals are fixed.
- Physical meaning
- Dividing by E makes a gap of 10 matter more in a cell expecting 20 than in one expecting 2,000. A big χ² means the mix of one category really changes across the other.
“Chi-square tells you which plan is different.” It only says the table as a whole is unlikely under “no link”. Look at the cells with the biggest (O − E)²/E, or the standardised residuals, to see where the difference is.
Watch it happen, one step at a time:
Maths and principle
Principle: under the null, each cell’s is roughly a standard normal; adding their squares gives a chi-square variable with df equal to the number of free cells.
Equations: ; ; df = ; .
In code: Excel CHISQ.TEST(observed, expected); Python scipy.stats.chi2_contingency(table, correction=False) and scipy.stats.chisquare(obs, f_exp); pandas pd.crosstab(df.plan, df.renewed) builds the table.
For 2 × 2 tables, the chi-square test equals the two-proportion z-test squared: χ² = z².
5. Worked example
The simulation’s starting numbers. Renewals by plan:
| Basic | Plus | Pro | Total | |
|---|---|---|---|---|
| Renewed | 120 | 90 | 70 | 280 |
| Left | 80 | 40 | 20 | 140 |
| Total | 200 | 130 | 90 | 420 |
- Expected renewed: Basic 200 × 280 ÷ 420 = 133.3; Plus 130 × 280/420 = 86.7; Pro 90 × 280/420 = 60. Expected left: 66.7, 43.3, 30.
- Contributions: (120 − 133.3)²/133.3 = 1.33; (90 − 86.7)²/86.7 = 0.13; (70 − 60)²/60 = 1.67; (80 − 66.7)²/66.7 = 2.67; (40 − 43.3)²/43.3 = 0.26; (20 − 30)²/30 = 3.33.
- χ² ≈ 9.38 with df = 2; p ≈ 0.009. Cramér’s V = √(9.38 ÷ 420) ≈ 0.15 (small to modest).
- Renewal rates: Basic 60 %, Plus 69 %, Pro 78 %. Higher plans renew more.
Report: “Renewal differs by plan (χ² = 9.4, df = 2, p = 0.009): 60 % on Basic, 69 % on Plus, 78 % on Pro.” Whether the plan causes renewal is another matter: committed customers may choose Pro.
6. Practice
- A die is rolled 60 times: counts 8, 9, 12, 11, 6, 14. Compute χ² for a fair die.
- A 2 × 2 table has 120 people in total, row totals 30 and 90, column totals 40 and 80. Find the expected count for the top-left cell.
- χ² = 2.1 with 3 df. Significant at 5 %? (The 5 % critical value for 3 df is 7.81.)
Answers
- Expected 10 each. (4 + 1 + 4 + 1 + 16 + 16) ÷ 10 = 4.2, with 5 df: p ≈ 0.52, consistent with fair.
- 30 × 40 ÷ 120 = 10.
- No: 2.1 is well below 7.81.
Try it on your own data: cross-tab and chi-square builds the table from a CSV and runs the test with Cramér’s V; the hypothesis tests tool does goodness of fit from pasted counts.
New words: chi-square test, goodness of fit, contingency table, expected count, test of independence, Cramér’s V, Fisher’s exact test.
- Expected count = row total × column total ÷ grand total.
- χ² = sum of (O − E)² ÷ E over all cells; df = (r − 1)(c − 1).
- Needs counts (not percentages) and expected counts of about 5 or more.
- A significant χ² says there is a link; look at the cells to see where.
Check yourself
Sources
- OpenIntro Statistics, 4th edition (Diez, Cetinkaya-Rundel, Barr), Chapter 6.3-6.4 Chi-square tests https://www.openintro.org/book/os/
- NIST/SEMATECH e-Handbook of Statistical Methods, 1.3.5.15 Chi-Square Goodness-of-Fit Test https://www.itl.nist.gov/div898/handbook/eda/section3/eda35f.htm
Your ticks stay in this browser. Save them as a file on the course home.
Updated 2026-11-22. Numbers in the examples are made up to show the idea unless a source is given. Simulations run in your browser; nothing you type is sent anywhere.