Free tool
A/B Test Sample Size Calculator
Stopped before it has enough visitors, an A/B test can pick a winner by pure chance. Before you launch, set the smallest lift that matters to you: the tool tells you how many visitors per variant you need to see it, and how many days it takes.
Sample size is the number of visitors each variant needs to detect a given gap between two conversion rates. It depends on the baseline rate, the target lift, the confidence level (often 95 %) and the power (often 80 %), which is the chance of detecting the gap if it exists.
Not Enough Traffic For Your Tests?
More qualified visitors shorten every test. Campaigns can send that traffic to the page you are testing, at the right volume.
20 minutes · No commitment · 100% confidential
How sample size is calculated
n per variant = (p1 × (1 − p1) + p2 × (1 − p2)) × (zα/2 + zβ)² ÷ (p2 − p1)²
With a 3 % conversion rate and a 15 % target lift, the variant must reach 3.45 %. At 95 % confidence and 80 % power you need 24,190 visitors per variant, 48,380 in total: just over 16 days at 3,000 visitors a day.
The target lift weighs the most. Halving the lift you want to detect multiplies the number of visitors by almost four, because the gap is squared in the formula. That is why a small site tests bold changes, and a large site can test details.
The formula is the standard one for comparing two proportions in a two-sided test, where the gap can go either way (Dalhousie University course). The result is a minimum: also run the test over full weeks, so every day of the week counts equally.
What the calculation saves you from
- Stopping a test as soon as a variant pulls ahead: early gaps move a lot while the sample is small.
- Launching a test your traffic cannot settle: if the calculation says three months, aim for a bigger lift or a busier page.
Frequently asked questions
How do I calculate sample size for an A/B test?
You need four numbers: the current conversion rate, the lift to detect, the confidence level and the power. With 3 %, 15 %, 95 % and 80 %, the formula gives 24,190 visitors per variant.
How long should an A/B test run?
Until each variant reaches its sample size: divide the total visitors needed by your daily traffic. 48,380 visitors at 3,000 a day gives just over 16 days, to round up to full weeks.
What is the difference with statistical significance?
Sample size is calculated before the test, to know how many visitors to wait for. Significance is calculated after, to know whether the observed gap is reliable: that is what the A/B test significance calculator does.
Growth partner
Positioning, acquisition, innovation: together we build a lasting growth system.
Théo MaupiléFreelance Paid Media
Google PartnerSuperMalter
