Kruskal–Wallis test calculator
Compare two to eight independent groups by their ranks: H, the p-value, effect size and which pairs differ.
Put each group on its own line, with numbers separated by commas, spaces or semicolons. You can start a line with a name and a colon. Use a point for decimals. Two to eight groups.
What does the Kruskal–Wallis test tell me?
It compares two or more independent groups without assuming that the values are normally distributed. All the values are ranked together from smallest to largest, and the test asks whether the groups’ ranks differ more than chance would explain. It is the rank-based counterpart of one-way ANOVA, and with exactly two groups it is the normal approximation of the Mann–Whitney U test without continuity correction (for SciPy’s two-group example, [1, 3, 5, 7, 9] against [2, 4, 6, 8, 10], H = 0.2727 is the square of the pairwise z = -0.5222, and the p-value is 0.6015). A small p-value says that at least one group tends to produce larger or smaller values than the others; it does not say which one. For that, follow up with Dunn’s test, below.
How is H calculated?
Rank all N values together, giving tied values the average of the ranks they share. If group i has n_i values and its ranks add up to R_i, then H = 12 ÷ (N(N + 1)) × Σ(R_i² ÷ n_i) − 3(N + 1). With ties, H is divided by the correction C = 1 − Σ(t³ − t) ÷ (N³ − N), where t is the number of values in each group of ties; R’s kruskal.test and SciPy’s kruskal report this corrected value. Under the null hypothesis H follows a chi-square distribution with k − 1 degrees of freedom (k = number of groups), and the p-value is its upper tail. With 2 degrees of freedom the tail is simply e^(−H/2).
Worked example: dust removal in three groups of people
The R help page for kruskal.test uses data of Hollander and Wolfe (1973): mucociliary efficiency, measured by the rate of dust removal, in 5 normal subjects (2.9, 3.0, 2.5, 2.6, 3.2), 4 subjects with obstructive airway disease (3.8, 2.7, 4.0, 2.4) and 5 with asbestosis (2.8, 3.4, 3.7, 2.2, 2.0). There are no ties. The rank sums are 36, 36, 33 with N = 14, so H = 12 ÷ (14 × 15) × (36² ÷ 5 + 36² ÷ 4 + 33² ÷ 5) − 45 = 0.7714. With df = 2 the p-value is e^(−0.7714 ÷ 2) = 0.68: no evidence of a difference. Because the groups are small, the calculator also lists the exact permutation p-value, 0.7108, found by checking all 252,252 ways to deal the ranks into groups of 5, 4 and 5; it agrees with the conclusion.
How are tied values handled?
Each group of equal values gets the average of the ranks it occupies, and H is divided by the tie correction above. SciPy’s documentation has a small example: groups [1, 1, 1], [2, 2, 2] and [2, 2] have rank sums 6, 18, 12. Without the correction H = 5; the correction is C = 0.7143 (two tie groups, of 3 and 5 values), so the corrected H = 7 and p = 0.0302, which are the statistic (7.0) and p-value (0.0302) SciPy prints. With few ties the correction barely matters; with many ties (ratings on a 1–5 scale, for instance) it can change H noticeably, so always use the corrected value.
When is the chi-square p-value not reliable?
The chi-square distribution is only an approximation, and it is poor when some groups are small: Wikipedia’s article says that if group sizes are below 5 the true distribution of H can differ quite a bit from chi-square, and the SciPy documentation asks for at least 5 measurements in each sample. When your data are small enough (at most 1,000,000 ways to assign the ranks to the groups), this calculator adds the exact permutation p-value: it counts the share of all rearrangements of the ranks whose H is at least as large as yours. With no ties this is the exact null distribution of H; with ties it is the permutation distribution given your tied ranks. In the example above the two p-values are 0.68 and 0.71.
Which effect size should I report?
Two measures based on H appear in the literature; Tomczak and Tomczak (2014) give both. Epsilon-squared is ε² = H ÷ ((N² − 1) ÷ (N + 1)), which simplifies to H ÷ (N − 1); it runs from 0 (no relationship) to 1 (perfect). Eta-squared based on H is η² = (H − k + 1) ÷ (N − k), the share of the variability in the ranks explained by the grouping; it can come out negative when H is below k − 1, and the R package effectsize then reports 0. For the dust-removal data ε² = 0.0593 and η² = 0 (H = 0.7714 is below k − 1 = 2). For R’s airquality data (ozone by month, N = 116, H = 29.267) ε² = 0.2545. The calculator shows both next to the p-value; report whichever you name explicitly.
Which groups differ? Dunn’s test
After a significant result, Dunn’s (1964) test compares every pair using the same pooled ranks: z = (R̄_i − R̄_j) ÷ σ, where R̄ are the mean ranks and σ² = [N(N + 1) ÷ 12 − Σ(t³ − t) ÷ (12(N − 1))] × (1 ÷ n_i + 1 ÷ n_j). The bracket is the variance of a rank with ties taken into account, as in the source code of the R package dunn.test. The p-value is two-sided, 2 × P(Z ≥ |z|), and the Bonferroni adjustment multiplies it by the number of pairs, m = k(k − 1) ÷ 2, capping at 1. The pairwise null hypothesis is that a value drawn from one group is larger than a value drawn from the other with probability one half. On R’s airquality ozone data, Dunn’s test with Bonferroni flags months 5 and 7, 5 and 8, 7 and 9, and 8 and 9, the same four pairs that the pairwise-Wilcoxon example on Wikipedia’s Kruskal–Wallis page flags. Choose the “Dunn’s pairwise comparisons” table above to see them for your data.
Common mistakes and limits
Do not use the test for the same people measured repeatedly: the groups must be independent, and repeated measures call for the Friedman test. Do not read a significant result as “the medians differ” unless the group distributions have the same shape; otherwise the test is about ranks, i.e. whether values from one group tend to be larger than values from another. Do not run many Mann–Whitney tests without adjusting for the number of comparisons; Dunn’s test with Bonferroni is built for that. And a non-significant p-value with small groups only means there was not enough data, not that the groups are equal: with groups of 4 or 5 the test has little power. If your data are roughly normal with similar spreads, the one-way ANOVA calculator uses more of the information in them (the values themselves, not only their ranks).
Sources: W. H. Kruskal and W. A. Wallis, Journal of the American Statistical Association 47 (1952) 583–621; O. J. Dunn, “Multiple comparisons using rank sums,” Technometrics 6 (1964) 241–252; documentation of R (stats::kruskal.test, with data of Hollander and Wolfe, 1973, and the airquality data set) and SciPy (scipy.stats.kruskal); Wikipedia, “Kruskal–Wallis test”; M. Tomczak and E. Tomczak, “The need to report effect size estimates revisited,” Trends in Sport Sciences 21(1) (2014) 19–25; the R package dunn.test (A. Dinno). Related: one-way ANOVA calculator, Mann–Whitney U test calculator, Wilcoxon signed-rank test calculator, effect size calculator and p-value calculator.