A/B Test Calculator
Overview
The A/B Test Calculator evaluates statistical significance for your A/B tests, whether you’re comparing conversion rates or continuous metrics. Input your sample sizes and results, instantly see p-values, confidence intervals, and effect sizes, calculate the minimum detectable effect for your test, and determine required sample sizes for achieving significance. Essential for making data-driven decisions about website changes, product features, and marketing campaigns.
Tips
- Always determine your sample size before running the test - post-hoc power analysis can be misleading
- A p-value below 0.05 indicates statistical significance but doesn’t guarantee practical significance
- Check the confidence interval width - narrow intervals give more precise estimates of the true effect
- Minimum detectable effect (MDE) shows the smallest difference you can reliably detect with your sample size
- For conversion rates, aim for at least 100-200 conversions per variant for stable estimates
- Don’t stop tests early just because they reach significance - this inflates false positive rates
- Consider both statistical and practical significance: a 0.1% lift might be significant but not worth implementing