A-Priori vs Post-Hoc Power Analysis: Sample-Size Planning Done Right

A-priori (or “prospective”) power analysis is done BEFORE collecting data. You specify the test, the smallest effect size you care about, alpha (typically 0.05), and a target power (typically 0.80). The output is the minimum sample size that gives the test at least that much chance of detecting that effect. This is the version reviewers, funders, and pre-registration platforms expect. Post-hoc power analysis comes in two flavors. Sensitivity power analysis is legitimate — given the sample size you have, what is the smallest effect you could detect at 80% power? That is a useful diagnostic. Observed (“achieved”) power, computed from the effect size actually estimated in your data, is misleading and widely discouraged. With p < α the observed power is always above 50% by construction; with p > α it is below 50%. It is a deterministic function of the p-value, not new information. If you must report something post-hoc, use a confidence interval for the effect.

For any parametric test, power depends on four quantities, and fixing three pins down the fourth. (1) Alpha — the type I error rate; smaller alpha lowers power. (2) Sample size n — power rises as n grows. (3) Effect size — power rises with effect size; the right one depends on the test (Cohen’s d for two-sample t, f for ANOVA, w for chi-square, r for correlation). (4) Power — typically targeted at 0.80 (so type II error rate β = 0.20). Pick three, solve for the fourth. A-priori solves for n. Sensitivity solves for the effect size. Doubling your sample size does not double your power; for a t-test, sample size needs to roughly quadruple to halve the minimum detectable effect.

You want to detect a Cohen’s d = 0.5 (medium) effect with alpha = 0.05 two-sided and power = 0.80. The required sample size per group from a non-central t calculation is approximately n = 64 per group, or 128 total. Drop the effect size to d = 0.3 and required n per group jumps to about 175 (350 total). Drop to d = 0.2 and you need about 394 per group (788 total). Notice the quadratic relationship: cutting d in half quadruples the required sample. To shortcut without software: n_per_group ≈ 16 / d² for power 0.80 and alpha 0.05 two-sided. d = 0.5 → n ≈ 64; d = 0.3 → n ≈ 178; d = 0.2 → n ≈ 400. The formula’s slight overestimate at large d is conservative and useful for planning.

You plan an ANOVA with k = 4 groups and an expected Cohen’s f = 0.25 (medium). At alpha = 0.05 and power = 0.80 the required total sample is about n = 180 (45 per group). Smaller f = 0.10 requires roughly n = 1,096 total. Cohen’s f relates to η² as f = √(η² / (1 − η²)), so f = 0.10 → η² = 0.010, f = 0.25 → η² = 0.059, f = 0.40 → η² = 0.138. Planning around η² instead of f tends to look more reasonable to subject-matter readers but the underlying calculation is identical.

You expect Pearson r = 0.30 (moderate) in the population. At alpha = 0.05 two-sided and power = 0.80 the required sample is about n = 84. For r = 0.20 the required sample doubles to about 194; for r = 0.10 it explodes to about 783. Use Fisher’s z transformation to do the calculation by hand: z = 0.5 × ln[(1 + r) / (1 − r)]. The standard error of z is 1/√(n − 3), and the required z* to achieve power = 0.80 at alpha = 0.05 two-sided is z* = (1.96 + 0.842) = 2.802. Solve for n: n = 3 + (2.802 / z_r)². For r = 0.30, z_r = 0.310, so n = 3 + (2.802 / 0.310)² = 3 + 81.7 = 84.7.

A chi-square test with df = 4 and Cohen’s w = 0.20 (small-to-medium) at alpha = 0.05 and power = 0.80 requires about n = 200 observations. Cohen’s w = √(Σ((p_i − p_0i)² / p_0i)), where p_0i is the null proportion in cell i and p_i is the expected alternative. Many planning exercises start with the alternative cell proportions and compute w directly. For df = 1 (a 2×2 table) the same w needs n ≈ 197; for df = 8, n ≈ 219. Df enters the calculation but more weakly than w or alpha.

The single most consequential decision is the planning effect size, and three principles apply. First, the smallest effect of PRACTICAL interest — what does your audience need the effect to be to act on it? — beats reporting-driven choices. Second, use prior meta-analytic estimates discounted by 25–50% for publication-bias inflation, because individual published studies systematically overestimate effects. Third, run the analysis at two or three plausible effect sizes and report the corresponding sample sizes; reviewers and funders prefer a sensitivity range to a single point estimate. Avoid copying Cohen’s rules of thumb (small/medium/large) for fields where they do not apply — in clinical pharmacology a “small” d = 0.2 may be the largest realistic effect, while in psychophysics a “medium” d = 0.5 is unusually small.

Pick the test (t, ANOVA, correlation, chi-square, regression), enter alpha, target power, and the planning effect size, and StatsIQ returns the required sample size. Flip the inputs to run sensitivity power analysis — given the n you can afford, what is the smallest effect detectable at 80% power? Observed/achieved power is deliberately not foregrounded because it is a deterministic function of the p-value and adds no information. This content is for educational purposes only.