How to Read ANOVA Output: Sum of Squares, Mean Square, F-Statistic, and Post-Hoc Tests

An ANOVA table answers one question: are the means of three or more groups different enough that the differences are unlikely to be caused by random sampling alone? The table partitions the total variability in your data into two sources — variability between groups and variability within groups — and then compares them using the F-statistic. If the between-group variability is large relative to the within-group variability, the group means are probably different. If it is small, the differences you see could easily be noise. Every row, column, and number in the table serves this comparison. Once you understand the logic, reading the output becomes mechanical rather than mysterious.

The sum of squares (SS) quantifies variability by adding up squared deviations. ANOVA output reports three sum of squares values. SS Between (also called SS Treatment or SS Factor) measures how much the group means differ from the overall grand mean. If all group means were identical, SS Between would be zero. Large values mean the groups are spread apart. The formula is SS Between = sum of n_i times (x-bar_i minus grand mean) squared, where n_i is the size of each group and x-bar_i is each group mean. SS Within (also called SS Error or SS Residual) measures how much individual observations within each group differ from their own group mean. This is the natural variability — the noise — that exists even if the treatment has no effect. The formula sums (x_ij minus x-bar_i) squared across all observations in all groups. SS Total is simply SS Between plus SS Within. It measures the total variability in the entire dataset, ignoring group membership entirely. This additive relationship is fundamental: the total variation in your data is completely decomposed into variation explained by group membership and variation left unexplained. When you report eta-squared (SS Between divided by SS Total), you are reporting the proportion of total variability that the grouping variable explains — the ANOVA equivalent of R-squared in regression.

Sum of squares alone cannot be compared directly because SS Between and SS Within are based on different numbers of independent pieces of information. You need to adjust for that, which is where degrees of freedom (df) and mean squares (MS) come in. Degrees of freedom for Between = k minus 1, where k is the number of groups. If you have 4 groups, df Between = 3. Degrees of freedom for Within = N minus k, where N is the total number of observations. If you have 40 observations across 4 groups, df Within = 36. Mean Square is the sum of squares divided by its degrees of freedom: MS Between = SS Between divided by (k minus 1), and MS Within = SS Within divided by (N minus k). Mean squares are variance estimates — MS Within estimates the common within-group variance (sigma squared), while MS Between estimates the within-group variance plus any additional variability due to real group differences. The F-statistic is the ratio: F = MS Between divided by MS Within. If the null hypothesis is true (all population means are equal), both MS values estimate the same thing (sigma squared), so their ratio should be close to 1. If the group means really do differ, MS Between will be inflated by the group differences, pushing F above 1. The further F is above 1, the stronger the evidence against equal means. You compare F to the F-distribution with df1 = k minus 1 and df2 = N minus k to get the p-value. Snap a photo of the ANOVA output and StatsIQ walks through each row, connecting the numbers to the underlying logic so you understand what F = 4.37 actually means in your specific problem.

A researcher compares test scores across three teaching methods (lecture, flipped, hybrid) with 20 students per group (N = 60). The output table is: Source | SS | df | MS | F | p-value Between | 840 | 2 | 420 | 5.25 | 0.008 Within | 4560 | 57 | 80 | — | — Total | 5400 | 59 | — | — | — Here is how to read every number. SS Between = 840 means the group means vary around the grand mean by a total squared deviation of 840. df Between = 2 because there are 3 groups minus 1. MS Between = 840 divided by 2 = 420 — this is the average squared deviation of group means, adjusted for the number of groups. SS Within = 4560 means individual scores within each group vary around their own group mean by a total of 4560. df Within = 57 because 60 total observations minus 3 groups. MS Within = 4560 divided by 57 = 80 — this estimates the common within-group variance. F = 420 divided by 80 = 5.25. The between-group variance estimate is 5.25 times larger than the within-group variance estimate. Under the null hypothesis, we would expect this ratio to be around 1. A ratio of 5.25 is large enough to produce p = 0.008. Since p = 0.008 is less than alpha = 0.05, we reject the null hypothesis. At least one teaching method produces a different mean score. Eta-squared = 840 divided by 5400 = 0.156 — the teaching method explains about 15.6% of the variability in test scores, which is a medium-to-large effect by Cohen's benchmarks. But we still don't know which methods differ from which. That requires post-hoc tests.

A significant ANOVA F-test tells you that at least one group mean differs from the others. It does not tell you which groups differ. To answer that question, you run post-hoc (after-the-fact) pairwise comparison tests. These tests compare every pair of group means while controlling for the increased risk of false positives from running multiple comparisons. Why not just run multiple t-tests? Because the more comparisons you run, the higher the chance of at least one false positive. With 3 groups, there are 3 pairwise comparisons. With 5 groups, there are 10. With 10 groups, there are 45. If each test uses alpha = 0.05, the probability of at least one false positive across all tests — the family-wise error rate — grows quickly. For 10 comparisons at alpha = 0.05, the family-wise error rate is about 40%, not 5%. Tukey's HSD (Honestly Significant Difference) is the most common post-hoc test. It controls the family-wise error rate at your chosen alpha while comparing all pairs of means. It calculates a critical difference: if the absolute difference between two group means exceeds the Tukey critical value times the standard error, those means are significantly different. Tukey HSD is the default choice when you want to compare all pairs. Bonferroni correction is simpler but more conservative: divide alpha by the number of comparisons. For 3 comparisons at alpha = 0.05, each comparison uses alpha = 0.05 divided by 3 = 0.0167. This controls the family-wise error rate but is more likely to miss real differences (lower power) than Tukey, especially with many groups. Dunnett's test is specialized for comparing each treatment group to a single control group (not all pairwise comparisons). It is more powerful than Tukey or Bonferroni when you only care about treatment-versus-control differences. The choice of post-hoc test should match your research question: Tukey for all pairs, Dunnett for treatment-vs-control, Bonferroni for a small number of planned comparisons.

ANOVA assumes three things about your data, and violating them can produce misleading results. Independence means the observations are not related to each other. Each data point comes from a different, independently selected subject. This assumption is violated by repeated-measures designs (the same person measured multiple times), which require repeated-measures ANOVA instead of one-way ANOVA. Normality means the data within each group are approximately normally distributed. ANOVA is robust to moderate violations of normality, especially with balanced designs (equal group sizes) and sample sizes above 15-20 per group. For severely non-normal data or small samples, the Kruskal-Wallis test is a non-parametric alternative that does not assume normality. Equal variances (homoscedasticity) means the spread of data is similar across groups. Levene's test checks this assumption: if Levene's test is significant (p less than 0.05), the variances are unequal. When variances are unequal, the standard F-test can produce inflated Type I error rates. Welch's ANOVA is the recommended alternative — it adjusts the degrees of freedom to account for unequal variances and does not require the homoscedasticity assumption. A practical approach: run Levene's test first. If it is non-significant, use standard ANOVA. If it is significant, switch to Welch's ANOVA. For post-hoc tests after Welch's ANOVA, use the Games-Howell procedure instead of Tukey HSD, since Games-Howell does not assume equal variances.