AB Test Significance & Statistics Calculator, Debunked

Conversion Rate

This is the number of conversions you expect to get for every visitor on your page. It is given as a percentage and calculated like this:

AB testing is the best way to make sure you increase your conversion rate in the long run.

Uplift

Uplift is the relative increase in conversion rate between page A and page B. It is possible to have negative uplift if your original page is more effective than the new one. It is calculated like this:

It is important to remember that this is an increase in the rate conversions, not in absolute sales.

AB Test Significance

Within AB testing statistics, your results are considered “significant” when they are very unlikely to have occurred by chance. Achieving statistical significance with a 95% Confidence Level means you know that your results will only occur by chance once in every 20 times.

P-Value

Your P-Value is the probability that your results have occurred as a result of random chance. If this number is lower than your “Alpha” value (which is just 100 minus your Confidence Level) then your results are significant.

A high P-Value means your results are not significant. It could be due to your sample size, the size of your Uplift, or the way your data is scattered.