A-B Split

What Is an A-B Split?

An A-B split test is a way to compare two versions of a marketing campaign, mobile application, website, or other measurable media to determine which performs better. In an A-B split test, users are divided randomly into a control group and a variation group.

These two sets of users are then shown essentially the same media with the exception of a single variable, for example, the size of an “Order Now” call to action button on an e-commerce page. The results are measured to determine the success of the variable.

The A-B split is also referred to as A/B testing, bucket tests, or split-run testing.

Key Takeaways

• An A-B split test is a way to compare two versions of a marketing campaign, mobile application, website, or other measurable media to determine which performs better.
• In an A-B split test, users are divided randomly into a control group and a variation group.
• These two sets of users are then shown essentially the same media with the exception of a single variable; the results are measured to determine the success of the variable.
• The A-B split is also referred to as A/B testing, bucket tests, or split-run testing.
• Though A-B split testing has long been used in marketing, the internet allows practitioners to design and deploy tests more quickly, which significantly accelerates the design iteration process.

Understanding an A-B Split

The A-B split has been used for decades in direct mail campaigns. It has also been successfully adapted to interactive media for testing e-mail blasts, banner advertisements, website, and mobile app functionality (in addition to other uses).

Audiences are divided into two groups: the control group and the variation group. For example, a newsletter publisher might wish to test the effectiveness of a call to action such as, “Subscribe within 48 hours to receive a 20% discount.”

The variation group receives the newsletter with the call to action, while no offer is made to the control group. This enables the publisher to determine whether the call to action is effective and whether the response is sufficient to justify the 20% discount.

Though A-B split testing has long been used in marketing, the internet allows practitioners to design and deploy tests more quickly, which significantly accelerates the design iteration process. A-B split testing can be performed continuously at relatively little cost, allowing for constant fine-tuning of marketing campaigns, website updates, and the development of online tools. In effect, A-B split testing results in actionable guidance on website optimization that utilizes data in decision-making.

Special Considerations

A-B Split Testing Steps

The following are steps to create and execute an A-B split test:

1. Collect data to identify opportunities for improvement.
2. Identify goals such as improving click-through rates or e-mail signups.
3. Form a hypothesis for an A-B split test idea.
4. Create a test variation with a single variable, such as the color of a key button.
5. Run the test to measure user interactions.
6. Analyze the data to determine whether the results are statistically significant to act upon.