A/B Testing – Split Glossary – Feature Flag Management Platform

A/B testing, otherwise known as split testing, is the process of testing two different versions of a web page or product feature in order to optimize conversion rate, or improve upon a certain business metric. The two versions can be very similar, with only a change in button color, or very different, with a total change in the way a feature behaves.

Deploy Continuously with Confidence – Sign up for a Split demo!

Speed up development cycles, reduce release risk, and focus your team on DevOps best practices that create maximum impact. Get a personalized demo today!

How does A/B testing work?

A/B testing is based on the scientific method, and the process is very similar. To start with, gather relevant data on your current features and see which ones have the most potential to improve key business metrics. After you have the baseline data, look at those features to see how customers are utilizing them, and hypothesize a variation which could improve it.

Since the next step is to build the new version, you’ll want to take improvements with similar expected user experience improvement and prioritize them by how easy they are to build. Then, pick a test to start with and build the new version. The old version will be what scientists call the “control” and what we’ll call Version A; the new version is the “experiment”: Version B.

With front end A/B testing, people typically assign Version A and Version B to different sets of users and measure which set of users, if either, had a higher conversion rate with statistical significance. But there is a different kind of A/B testing which happens at a much deeper level.

A/B testing with feature flags

While typical A/B testing happens on the front end, choosing which version of the page show to website visitors, there is a way to A/B test your product features as well: using feature flags.

Feature flags allow development teams to release a feature to only a subset of users, which satisfies the necessary step of creating two versions of a feature. All that’s left to do is to integrate the team’s analytics platform with the feature flag management system, such that the team can correlate the users’ behavior with which version of the feature they used.

After these things are done, the A/B testing process can be used to find the expected user experience change when any new feature or code change is implemented. Development teams can then look at this information and adjust the feature accordingly: if the change is significantly negative, they can find out what’s wrong roll back the feature so it performs as it did before running the test; if it’s positive, they can release the product feature to a larger percentage of their customers.

Get Split Certified

Split Arcade includes product explainer videos, clickable product tutorials, manipulatable code examples, and interactive challenges.

Deliver Features That Matter, Faster. And Exhale.

Split is a feature management platform that attributes insightful data to everything you release. Whether your team is looking to test in production, perform gradual rollouts, or experiment with new features–Split ensures your efforts are safe, visible, and highly impactful. What a Release. Get going with a free account, schedule a demo to learn more, or contact us for more information.