Permutation Test Settings
Group Distributions
Null Distribution from Permutations
Understanding Permutation Tests
Permutation tests are a powerful non-parametric approach to hypothesis testing:
- Null hypothesis: The two groups come from the same distribution (labels are exchangeable)
- Permutation process: Randomly reassign observations to groups, calculate test statistic each time
- Null distribution: The distribution of test statistics under all possible permutations
- P-value: Proportion of permuted statistics as extreme or more extreme than observed
- Advantages: No distributional assumptions, exact for small samples, robust to outliers
- Test statistics: Mean difference is intuitive, t-statistic accounts for variance, median is robust
- Alternative hypotheses: Two-sided tests both tails, one-sided tests only one direction