Test result

link copied to clipboard!

Variation B’s observed conversion rate (

) was

higher

than variation A’s conversion rate (

). You can be

95

% confident that this result is a consequence of the changes you made and not a result of random chance.

The observed difference in conversion rate (

) isn’t big enough to declare a significant winner. There is no real difference in performance between A and B or you need to collect more data.

Possible SRM alert

Assuming you intented to have a 50% / 50% split, a Sample Ratio Mismatch (SRM) check indicates there might be a problem with your distribution.

The expected distributions of variation A and B.

Conversion Rate Control

Conversions A / Visitors A

Conversion Rate B

Conversions B / Visitors B

Relative uplift in Conversion Rate

CRB – CRA / CRA

Observed Power

p value

Z-score

( CRB – CRA ) / SEdifference

Standard error A

( CRA * (1-CRA ) / VisitorsA)1/2

Standard error B

( CRB * (1-CRB ) / VisitorsB)1/2

Std. Error of difference

SEdifference = ( SEA2 + SEB2 )1/2