Test Type
Parameters
Power Curve
Type I vs Type II Error Trade-off
Understanding Statistical Power
Statistical Power: The probability of correctly rejecting the null hypothesis when it is false (i.e., detecting a true effect).
Key Concepts:
- Type I Error (α): False positive - rejecting a true null hypothesis
- Type II Error (β): False negative - failing to reject a false null hypothesis
- Power = 1 - β: The probability of avoiding a Type II error
- Effect Size: The magnitude of the difference you want to detect (Cohen's d)
Effect Size Reference (Cohen's d)
- Small (0.2): Difficult to detect, subtle difference (e.g., height difference of 0.5 inches)
- Medium (0.5): Noticeable to careful observer (e.g., IQ difference of 7.5 points)
- Large (0.8): Obvious to casual observer (e.g., height difference of 2 inches)
Sample Size Formula (two-sample t-test): n ≈ 2(zα/2 + zβ)² / d²