A/B Test Significance Calculator
Is your A/B test result statistically significant?
🅰️ Control (A)
🅱️ Variant (B)
A conversion rate
0.00%
B conversion rate
0.00%
Uplift (relative)
0%
P-value
—
Enter numbers to see verdict
📚 Learn more — how it works, FAQ & guide Click to expand
Learn more — how it works, FAQ & guide
Click to expand
Free A/B test significance calculator — z-test for conversions
Calculate if your A/B test result is statistically significant. Two-proportion z-test with pooled variance. See p-value, confidence intervals, and clear verdict.
How to use this tool
- 1
Enter visitors + conversions
For both control (A) and variant (B): how many visitors and how many converted.
- 2
Check significance
See conversion rates, uplift %, p-value, confidence interval, statistical significance.
- 3
Interpret result
p < 0.05 = statistically significant. 95%+ confidence means the result isn't random noise.
Interpretation guide
- p < 0.01: very strong evidence against null (99%+ confidence)
- p < 0.05: standard "significant" threshold (95% confidence)
- p < 0.10: weak evidence — don\'t call it significant
- p ≥ 0.10: no significant difference detected
Frequently Asked Questions
What is statistical significance?
The probability that your observed difference is NOT random chance. p < 0.05 = less than 5% chance the difference is random — typically the threshold to consider a test "significant".
What sample size do I need?
Depends on your baseline conversion rate + minimum detectable effect. Rough rule: 1,000-10,000 per variant for typical web A/B tests. Low conversion rates (1%) need more samples; high rates (20%) need fewer.
What if my test is "not significant"?
Three options: (1) Run longer until sample size is adequate — but don't peek and stop early. (2) Accept the null hypothesis — the variants perform equally. (3) Bigger change next time — tiny differences need enormous samples to detect.
Z-test vs t-test vs chi-square?
For proportions (conversion rates), z-test with pooled variance is standard and what we use. For continuous metrics (revenue per visitor), t-test is more appropriate. This tool focuses on conversion-rate tests.
What's confidence interval?
The range where the true difference likely falls. "B is 15% better (CI: 8% to 22%)" means the real uplift is probably between 8-22% — we just don't know exactly where.
You might also like
🔒
100% Privacy. This tool runs entirely in your browser. Your data is never uploaded to any server.