Chi-square test of independence

What does the test of independence check?

Whether two categorical variables are related, using a table of counts — for example, whether commute distance depends on job type. The null hypothesis is that the variables are independent; the alternative is that they are dependent. The test is always right-tailed: only a large χ² counts against independence (OpenStax, Introductory Statistics 2e, section 11.3).

How are expected counts found?

Expected = row total × column total ÷ grand total, for each cell. χ² adds (observed − expected)² ÷ expected over all cells, with (rows − 1) × (columns − 1) degrees of freedom. The expected table is what the counts would look like if the row a person falls in told you nothing about the column; once the row and column totals are fixed, only that many cells are free to vary.

Worked example: volunteer hours

OpenStax (Example 11.6) cross-classifies 839 people by type of volunteer (community college students, four-year college students, nonstudents) and weekly hours (1–3, 4–6, 7–9). The row totals are 255, 290, 294 and the column totals 298, 379, 162. The first expected count is 255 × 298 ÷ 839 = 90.57, against 111 observed. Adding the nine terms gives χ² = 12.99 with (3 − 1)(3 − 1) = 4 degrees of freedom and p = 0.0113. The 5% critical value is 9.49, so hours volunteered and volunteer type are dependent. The largest single term, 4.61, comes from row 1, column 1. The association is weak: Cramér’s V = √[χ² ÷ (N × (smaller of rows and columns − 1))] = 0.09, on a scale from 0 to 1.

When is the test reliable?

A common rule is that every expected count should be at least 5 (OpenStax and the SciPy documentation both state it). The calculator warns you when one is smaller. In the example the smallest expected count is 49.2. For a 2 × 2 table with small counts, use Fisher’s exact test instead. SciPy’s chi2_contingency also applies Yates’ continuity correction to tables with one degree of freedom by default; this calculator reports the plain Pearson statistic, so a 2 × 2 result can differ slightly from software that corrects.

How do I enter the table?

One row per line, the counts separated by commas or spaces. Every row needs the same number of columns.

Common mistakes with the test of independence

First, entering percentages, rates or averages instead of counts: χ² depends on how many people were counted, so the table must hold real frequencies. Second, merging or dropping categories after seeing the result to get a smaller p-value; decide the categories first. Third, reading a small p-value as a strong or causal link. The p-value only says the variables are unlikely to be independent, and with thousands of observations a trivial difference becomes significant; look at Cramér’s V and at which cells depart most from their expected counts. Fourth, using the test on matched or repeated observations of the same people, which breaks the independence of the observations. A row or column that sums to zero cannot be tested, so remove it.

What is the difference between a test of independence and a test of homogeneity?

The arithmetic is the same; the sampling differs. In a test of independence one sample is classified two ways. In a test for homogeneity, OpenStax section 11.4, you take separate samples from separate populations, for example men and women, and ask whether they share the same distribution of one categorical variable. The null hypothesis is then that the distributions match. For a 2 × 2 table the statistic equals the square of the two-proportion z-test: for 20 of 200 against 12 of 200, χ² = 2.17, which is exactly z².

Sources: OpenStax, Introductory Statistics 2e, sections 11.3 (Test of Independence, Example 11.6) and 11.4 (Test for Homogeneity); SciPy documentation for scipy.stats.chi2_contingency and scipy.stats.contingency.association (Cramér’s V). Related: chi-square goodness-of-fit calculator, Fisher’s exact test calculator, two-proportion z-test, critical value calculator and relative risk calculator.