Free calculator
How long should your A/B test run?
Enter your conversion rate, the smallest lift worth finding and your daily traffic. You'll see how many visitors each variation needs and how many days that takes, planned for results you check every day.
Your test needs
25 days
- visitors per variation
- 61,330
- visitors in total
- 122,660
Enough to find a change from 5% to 5.5%.
Plan for 4 full weeks (28 days), so every weekday counts the same.
Run this test in LiftCanaryAt 80% power, a real lift this size is found 8 times in 10. Estimated with LiftCanary's own always-valid test (mSPRT), corrected for the number of variations.
How to read it
What goes into the number
Check it daily, it stays valid
Classic calculators assume you look once, at the end. Look at a classic test every day and it calls a false winner about 30% of the time after 40 looks. This one plans for LiftCanary's always-valid statistics: you can check every day, and stop early when the real lift is bigger than the one you planned for. The trade-off is more visitors than a classic calculator asks for, a number that only holds if you never peek.
The lift you pick matters most
Halve the smallest lift worth finding and you need about four times the visitors. Pick the smallest change you'd actually ship, not the one you hope for.
Every variation costs traffic
Each extra variation needs its own visitors, and the confidence is corrected for the extra comparisons, so no variation wins by luck. Test your two or three best ideas, not ten.
Too long? Change the test, not the math
Test a bolder change, put more of your traffic in, or measure a goal closer to the change, like add to cart instead of purchase. In LiftCanary, CUPED uses what returning visitors did before the test to cut the noise, which often gets you there sooner than this estimate.
Plan it where you run it
When you set up an experiment in LiftCanary, it estimates this from your real traffic, and the results stay valid every time you look.