All lessons

Module 6: Hypothesis tests · Lesson 6.5 · about 25 min

Chi-square tests for categories

Tests for counts in categories: goodness of fit (does the mix match what we expect?) and independence (are two categorical columns related?). Expected counts, the χ² sum, degrees of freedom, the small-count rule and Cramér's V.

Key points, meanings and formulas on one page: choose "Save as PDF" in the print box.

Module 6 · about 25 minutes · school maths. You’ll be able to run chi-square goodness-of-fit and independence tests, check their small-count rule, and report the strength of the relationship.

Chi-square compares the counts you see with the counts you would expect if the categories were unrelated, and adds up the squared surprises.

In everyday terms: a bag of sweets is meant to be one third each of red, green and yellow. You count yours. Chi-square measures how far your counts are from a fair third each, allowing for the fact that a small bag never splits perfectly.

1. Tests for counts

Many business questions are about categories, not averages:

  • Is churn related to the plan a customer is on?
  • Do complaints split across regions the way sales do?
  • Is the 50/50 split in our A/B test really 50/50? (Lesson 7.6)

The data is a table of counts. The chi-square (χ²) test compares observed counts O with the counts E you would expect under the null hypothesis:

χ2=∑cells(O−E)2E

Each cell adds its squared gap, scaled by how big a count was expected (a gap of 10 matters more when 20 were expected than when 2,000 were).

2. Goodness of fit

One categorical column, compared with expected shares. Degrees of freedom = categories − 1.

Example: complaints by region were 50, 30, 20 (n = 100), and sales shares are 40 %, 40 %, 20%. Expected counts: 40, 40, 20. χ² = 100/40 + 100/40 + 0 = 5.0 with 2 df: p ≈ 0.082. The complaint mix doesn’t differ clearly from the sales mix.

3. Independence (contingency tables)

Two categorical columns cross-tabulated. Under independence, each cell’s expected count is

Eij=rowi total×columnj totalgrand total

and df = (rows − 1) × (columns − 1).

Try it: edit the counts
BasicPlusPro
Renewed
Left
χ² statistic
degrees of freedom
p-value
Cramér's V (strength)

Expected counts (outlined) are what each cell would hold if renewal had nothing to do with the plan: row total × column total ÷ grand total. χ² adds up (observed − expected)² ÷ expected over every cell. Large gaps, relative to what is expected, give a large χ² and a small p-value. Made-up numbers.

Edit the counts and watch the expected (outlined) bars and the χ² change. Make the Pro column’s renewals match the others and the p-value climbs.

4. Strength and the small-count rule

Cramér’s V turns χ² into a 0-to-1 strength measure: V=χ2/(n(k−1)), where k is the smaller of rows and columns. Rough guide for 2-column tables: 0.1 small, 0.3 medium, 0.5 large. With big data a tiny V can be “significant”; report V too.

Small counts: the χ² approximation needs expected counts of at least about 5 in every cell. If not, combine sparse categories or use Fisher’s exact test (for 2 × 2 tables).

A significant result says the columns are related, not how. Look at which cells have the largest (O−E)2/E contributions, or at row percentages, to describe it.

Observed O
The counts in each cell of your table.
Expected E
Row total × column total ÷ grand total: the count if there were no link.
χ² = Σ (O − E)² ÷ E
The total squared surprise, scaled by what was expected.
df = (rows − 1)(cols − 1)
How many cells are free to vary once totals are fixed.
Physical meaning
Dividing by E makes a gap of 10 matter more in a cell expecting 20 than in one expecting 2,000. A big χ² means the mix of one category really changes across the other.

“Chi-square tells you which plan is different.” It only says the table as a whole is unlikely under “no link”. Look at the cells with the biggest (O − E)²/E, or the standardised residuals, to see where the difference is.

Watch it happen, one step at a time:

The table. Renewals by plan: Basic 120 of 200, Plus 90 of 130, Pro 70 of 90. Overall 280 of 420 = 2/3.
Expected. If plan did not matter, each would renew 2/3: 133.3, 86.7, 60 renewals; 66.7, 43.3, 30 leavers.
Surprise per cell. (O − E)²/E: 1.33, 0.13, 1.67, 2.67, 0.26, 3.33.
Add. χ² ≈ 9.4 with (2 − 1)(3 − 1) = 2 df.
Read. p ≈ 0.009: renewal depends on plan. The biggest surprises are Basic leaving more and Pro leaving less.
Maths and principle

Principle: under the null, each cell’s (O−E)/E is roughly a standard normal; adding their squares gives a chi-square variable with df equal to the number of free cells.

Equations: χ2=∑(O−E)2/E; Eij=RiCj/N; df = (r−1)(c−1); V=χ2/(N(k−1)).

In code: Excel CHISQ.TEST(observed, expected); Python scipy.stats.chi2_contingency(table, correction=False) and scipy.stats.chisquare(obs, f_exp); pandas pd.crosstab(df.plan, df.renewed) builds the table.

For 2 × 2 tables, the chi-square test equals the two-proportion z-test squared: χ² = z².

5. Worked example

The simulation’s starting numbers. Renewals by plan:

Basic Plus Pro Total
Renewed 120 90 70 280
Left 80 40 20 140
Total 200 130 90 420
  • Expected renewed: Basic 200 × 280 ÷ 420 = 133.3; Plus 130 × 280/420 = 86.7; Pro 90 × 280/420 = 60. Expected left: 66.7, 43.3, 30.
  • Contributions: (120 − 133.3)²/133.3 = 1.33; (90 − 86.7)²/86.7 = 0.13; (70 − 60)²/60 = 1.67; (80 − 66.7)²/66.7 = 2.67; (40 − 43.3)²/43.3 = 0.26; (20 − 30)²/30 = 3.33.
  • χ² ≈ 9.38 with df = 2; p ≈ 0.009. Cramér’s V = √(9.38 ÷ 420) ≈ 0.15 (small to modest).
  • Renewal rates: Basic 60 %, Plus 69 %, Pro 78 %. Higher plans renew more.

Report: “Renewal differs by plan (χ² = 9.4, df = 2, p = 0.009): 60 % on Basic, 69 % on Plus, 78 % on Pro.” Whether the plan causes renewal is another matter: committed customers may choose Pro.

6. Practice

  1. A die is rolled 60 times: counts 8, 9, 12, 11, 6, 14. Compute χ² for a fair die.
  2. A 2 × 2 table has 120 people in total, row totals 30 and 90, column totals 40 and 80. Find the expected count for the top-left cell.
  3. χ² = 2.1 with 3 df. Significant at 5 %? (The 5 % critical value for 3 df is 7.81.)
Answers
  1. Expected 10 each. (4 + 1 + 4 + 1 + 16 + 16) ÷ 10 = 4.2, with 5 df: p ≈ 0.52, consistent with fair.
  2. 30 × 40 ÷ 120 = 10.
  3. No: 2.1 is well below 7.81.

Try it on your own data: cross-tab and chi-square builds the table from a CSV and runs the test with Cramér’s V; the hypothesis tests tool does goodness of fit from pasted counts.

New words: chi-square test, goodness of fit, contingency table, expected count, test of independence, Cramér’s V, Fisher’s exact test.

  • Expected count = row total × column total ÷ grand total.
  • χ² = sum of (O − E)² ÷ E over all cells; df = (r − 1)(c − 1).
  • Needs counts (not percentages) and expected counts of about 5 or more.
  • A significant χ² says there is a link; look at the cells to see where.

Check yourself

1What is the expected count in a cell of a contingency table, if the two variables are independent?
2A table has 3 rows and 4 columns. How many degrees of freedom does the chi-square test of independence have?
3Several expected counts are below 5. What should you do?

Sources

Your ticks stay in this browser. Save them as a file on the course home.

Updated 2026-11-22. Numbers in the examples are made up to show the idea unless a source is given. Simulations run in your browser; nothing you type is sent anywhere.