10 best A/B testing tools for websites and mobile apps in 2023 | Smartlook Blog

A/B testing tools allow you to compare two versions of a webpage or mobile app against each other to see which one converts better. Having this ability to run experiments can help you:

• Optimize key user flows such as sign-ups, onboarding tutorials, and checkouts. 
• Test all kinds of design and copy elements to see how they affect user behavior.
• Make data-driven decisions, instead of blindly implementing design or copy changes and hoping they lead to better results.

But despite these massive benefits, A/B tests can easily backfire, leading you to incorrect conclusions. This can happen when you don’t run clean experiments, for example, by not waiting for a large enough sample size or introducing more changes than intended in a variant.

Now, most A/B testing tools help you avoid some of these scenarios. For instance, they tell you when a test reaches statistical significance, so you don’t make conclusions before that. 

However, A/B testing tools can’t show you how users interact with each variant you’re testing. This means that a test’s outcome can actually be misleading when multiple changes are introduced in a variant (on purpose or by accident) since you only see which variant performed better but not why. 

Put simply, most A/B testing tools provide no way of verifying the accuracy of your experiments. As a result, you may end up spending engineering hours and money implementing design changes that actually perform worse than the original.

We’ll start this post by showing how to overcome this issue with Smartlook — our product analytics and visual user insights platform. Specifically, you’ll learn how to combine Smartlook with popular A/B testing tools to verify the accuracy of your experiments. We’ll also explore ways to generate good hypotheses for new tests and go over the benefits of mobile app A/B testing.

Lastly, we’ll review 10 A/B testing tools for running website and mobile app experiments:

How to verify the accuracy of your A/B tests with Smartlook

Again, the ability to validate the accuracy of your A/B tests is essential, as it prevents you from spending time and money on implementing variants that actually hurt your UX and conversion rates.

For example, say you’re running an A/B test to determine which call to action (CTA) button copy results in more users making a purchase in your e-commerce store. Using an A/B testing tool, you create a variant that includes the new phrase you want to test out. 

But in the process, you also introduce a second change to the variant by merging the “First Name” and “Last Name” form fields into one “Name” field. When you evaluate this experiment’s results in your A/B testing tool, you’ll only see which variant performed best. 

But since there are actually two changes being tested, you can’t be certain which one is responsible for the outcome. If the new variant does perform better, was it because the new CTA button was more compelling than the original? Or was it because more customers completed the form when they were required to fill out one less field?

Now, you can avoid such issues by only testing one variable. In fact, changing only one thing at a time is a fundamental rule for running clean tests. However, multiple changes can get introduced to a variant without you even knowing it. 

For example, an A/B test may trigger a bug for users on a particular device or browser, which alters their behavior. Or someone in your organization may make a seemingly minor update on the live version of the page you’re testing, which serves as the control group for your test. 

Such inconsistencies happen all the time. In A/B testing terms, they’re called “confounding variables”. But since A/B testing tools can’t show you how users interact with each variant, they usually remain unnoticed. Even if you suspect something’s wrong, you can’t easily verify or disprove your hunch without running the experiment again and hoping for more reliable results.

Put simply, you need a way to check the accuracy of your experiments during the testing process, so you can be confident when implementing a specific variant. This is where Smartlook comes in.

Thanks to its session recording feature, Smartlook can show you exactly how users interact with your site or app. For example, you can see how a mobile app session recording looks in the image below.

The center shows a replay of the UI exactly as the user saw it, minus the system keyboard area, which isn’t recorded in order to protect user privacy when entering passwords. Below the session recording, there’s a dotted green line, representing the session timeline with all user interaction events that occurred. Lastly, on the left, you have additional details about the session, such as when it took place and on what device.

When it comes to A/B testing, you can use Smartlook’s session recordings to watch how users interact with each variant you’re testing. 

Let’s go back to our example of accidentally introducing more than one variable in an experiment. As we said, an A/B testing tool won’t be able to show you such an inconsistency. But with Smartlook, you’d immediately notice the differences when watching the session recordings for each variant.

To enable this capability, you first need to get the experiment and variation IDs from your A/B testing tool. For instance, in Optimizely, these IDs are located in the “API Names” menu, as described in this article.

Once you have the IDs, Smartlook’s Identify API lets you look up session recordings based on them. 

For example, say you want to watch how users interacted with “Variant B” of a specific experiment. In that case, you can locate the experiment and variant IDs in your A/B testing tool and find the session recordings in Smartlook with the Identify API.

You can learn more about this process in our articles on using Smartlook with:

Note: We’ve shown examples of the A/B testing tools that most of our customers use. However, the same process should work for any tool that provides you with variation and experiment IDs.

In short, using your A/B testing tool along with Smartlook’s session recordings lets you:

• Verify the accuracy of your experiments’ results by watching exactly how users interact with each variant.
• Confidently spend resources on implementing a variant because you can see why it performs better.

How to generate ideas for what to test by analyzing user behavior data

Besides validating your results, A/B testing tools have another blindspot — they don’t tell you what you should test. And since there are countless ways users interact with a site or app, coming up with good test hypotheses can be challenging.

While there are different methods you can use to generate ideas for new tests (like surveys and usability tests), the best way to fuel your experiments is with user behavior data (i.e.,information on how users interact with your site or app). This data can show you which pages users are visiting, what actions they’re taking, where they’re struggling, and much more.

Besides session recordings, Smartlook has three key analytics features that can help you generate A/B testing hypotheses:

1. Event tracking: Event tracking lets you monitor business-critical user actions over time, like account logins, button clicks, or in-app purchases. Smartlook tracks tons of events automatically and also lets you define custom events using JavaScript.
2. Funnel analysis: Funnel analysis shows you how many people move from one step to the next through key user flows (like onboarding and checkout) as well as the conversion rates between each step. 
3. Heatmaps: Heatmaps provide an overview of a typical user’s behavior on a page or mobile app screen. 

On its own, this combination of quantitative and qualitative analytics features provides tons of insights into your visitors’ behavior. We’ve shown how to get the most out of these features in our article on tracking user activity on websites, so we won’t go over all the details here.

For now, it’s important to note that you can use Smartlook’s Identify API to get even more valuable information about your A/B tests and their impact on the entire customer journey.

Let’s go back to our example of changing the CTA buttons on your e-commerce store’s homepage. This change can affect users’ behavior after they leave the homepage, so it’s a good idea to trace its impact on the whole checkout process.

First, Smartlook can map the entire checkout flow to a funnel, as we’ve shown in our funnel analysis article. This funnel shows you how many people move from one step to the next (e.g., from a product page, to adding an item to their cart, to reaching the checkout page), as well as where most of them drop off. 

Then, you can use the Identify API to group sessions of the A/B test with specific parameters and save them as filters. These can be used to break down your funnel, based on which homepage variant users initially landed on. 

As a result, you can see how your changes affected the entire purchase journey, not just users’ actions on your homepage. For example, you’d be able to see whether Variant A or Variant B led to more funnel drop-offs at each step, as well as which one resulted in more purchases. From here, you can create new A/B testing variants, based on the insights you got.

The importance of mobile A/B tests and validating their results

So far we’ve mostly explored A/B testing in the context of websites. However, A/B testing mobile apps is just as important.

Mobile users behave differently than website users. This means insights gathered from website A/B tests often don’t translate to mobile, so your app’s copy and design changes should be tested separately.

Additionally, mobile app updates require a lot of engineering resources, unlike website ones, which can often be made without coding. If changes aren’t tested beforehand or if their results haven’t been validated, the implementation of a mobile app variant that doesn’t perform better can cost you a lot of time and money (in addition to lost conversions).

That’s why testing and validating your mobile app’s changes is crucial. However, there’s a big issue here — very few tools actually have session recording capabilities for mobile. And as we showed above, session recordings are the primary tool for validating your experiments’ results.

Fortunately, Smartlook has SDKs for 10 native mobile app platforms, frameworks, and engines — including iOS, Android, React Native, Flutter, Unity, and Unreal Engine. 

You can set up Smartlook on your mobile app, then use a mobile A/B testing tool to conduct mobile A/B tests, and use Smartlook to verify the accuracy of the test results using session recordings.

Additionally, Smartlook’s other analytics features, including event tracking, funnel analysis, and heatmaps are also available for mobile apps. You can use them to analyze key moments in your app, like user onboarding, retention (via our Retention Tables), user engagement metrics, and in-app purchase flows. 

Again, gathering behavior data for your app can give you tons of experiment ideas. For example, say you build a funnel to analyze your app’s onboarding tutorial and see that most users drop off between two specific steps. You can try A/B testing different versions of these steps while using Smartlook to validate the results and see which one reduces funnel drop-offs.

Lastly, for a more in-depth look into Smartlook’s analytics capabilities for your mobile app, check out our article on mobile app tracking.

If you want to see how Smartlook works in detail, start a free 30-day, full-featured trial — no credit card required.

10 best A/B testing tools for websites and mobile apps

In this section, we’ll look at 10 A/B testing software tools (sometimes referred to as split testing or bucket testing tools) categorized by:

• The platforms you can use them on (i.e., web or mobile).
• Their key features.
• Their pricing.

• Supported platforms: Web only. Can be used on mobile apps only through Firebase A/B testing.
• Key features: A/B testing,multivariate testing,split URL testing, and server-side experiments.
• Pricing: Google Optimize is free. There’s a paid enterprise version (Optimize 360) but there’s no publicly available pricing.

Google Optimize is one of the most popular A/B testing solutions out there. It’s completely free and designed to work with other popular Google products like Google Analytics, Google Ads, and Firebase.

Note: See how to integrate Google Optimize with Smartlook in this article.

• Supported platforms: Mobile apps only.
• Key features: A/B testing, remote config experiments, messaging experiments, and in-app messaging experiments.
• Pricing: Firebase A/B testing is free.

Firebase is an app development platform created by Google. Its A/B testing module can help you experiment with changes to your app’s features, UI, or engagement campaigns.

Note: See how to integrate Firebase A/B Testing with Smartlook in this article.

• Supported platforms: Web and mobile.
• Key features: A/B testing, multi-page experiments (also called funnel tests), marketing planning, and a content management system (CMS).
• Pricing: There’s a free plan with the most basic functionalities. Paid plans are custom, depending on which Optimizely products you want to use.

Optimizely is a digital experience platform. It comes with A/B and multivariate testing capabilities, as well as a CMS, website personalization features, feature toggle capabilities, and more.

Note: See how to integrate Optimizely with Smartlook in this article.

• Supported platforms: Web and mobile.
• Key features: A/B testing, multivariate testing, and split URL testing.
• Pricing: Available upon request. A free 14-day trial is also available.

Visual Website Optimizer (VWO) is an all-in-oneA/B testing platform powered by SmartStats — a Bayesian statistics engine. Besides its testing features, VWO also offers analytics, server-side testing, personalization tools, and more. 

One of VWO’s most popular features is its intuitiveVisual Editor, which is a “what you see is what you get” tool (commonly referred to as WYSIWYG editor). You can use it to change page layouts via actions like drag and drop (i.e., without writing code). 

For more advanced users, VWO also offers a Code Editor, which lets you make changes directly in a site’s HTML, CSS, and JavaScript.

• Supported platforms: Web and mobile.
• Key features: Client-side experiments, AI-powered personalization, and engagement widgets.
• Pricing: Available upon request.

AB Tasty is a customer experience optimization platform. It has a broader focus than just A/B testing, with additional personalization, feature management, segmentation via machine learning algorithms, and product optimization capabilities.

• Supported platforms: Web and mobile.
• Key features: A/B and multivariate testing, AI-powered automations, and omnichannel personalization.
• Pricing: Available upon request.

Adobe Target is an experimentation platform that’s part of Adobe Experience Cloud. Like the entire experience cloud, Adobe Target is built for enterprises, with a focus on omnichannel user experiences and running tests across thousands or even millions of users.

• Supported platforms: Web and mobile.
• Key features: Testing and targeting, real-time personalization, advanced targeting, and predictive insights.
• Pricing: Available upon request.

Maxymiser is a testing and optimization tool that was acquired by Oracle in 2015. Its main focus is putting experimentation and personalization into the hands of marketers by removing the need for development resources.

• Supported platforms: Web only.
• Key features: A/B testing, web personalization, surveys, and advanced segmentation.
• Pricing: Free plan is available. Paid plans start at $99/month with a 30-day trial.

Omniconvert is a conversion rate optimization (CRO) software that comes with three distinct modules:

1. Revive, which includes the fundamental CRO features like A/B testing and segmentation.
2. Reveal, which is an e-commerce customer analytics platform.
3. Survey, which comes with various ways to get customer feedback, like click surveys and exit surveys.

• Supported platforms: Web and mobile.
• Key features: A/B testing, real-time personalization, and feature management.
• Pricing: Available upon request.

Kameleoon is an A/B testing and personalization platform that helps teams increase conversions. Similar to the previous entry, this platform also offers various features, bundled together under three modules:

1. A/B Testing, which allows unlimited A/B and multivariate test variations.
2. Full Stack, which helps rollout features to targeted audiences, update them remotely, and roll them back with automatic alerts.
3. Personalization, which helps increase user engagement by providing visitors with tailored experiences across channels and devices.

• Supported platforms: Web only.
• Key features: Landing page builder, A/B testing, pop-ups and an AI-powered copywriting generator.
• Pricing: Starts at $90/month with a free 14-day trial.

Unbounce is a unique entry to this list, as its focus isn’t solely on experimentation, but on building landing pages from scratch or with the help of pre-made templates. 

The user-friendly landing page builder has a built-in A/B testing tool, so you can create landing pages and A/B test different variants in one tool. Additionally, Unbounce also has a Smart Traffic feature, which automatically sends visitors to the landing page variant most likely to convert them.

Validate your A/B test results and analyze your users’ behavior with Smartlook

It’s easy to incorporate Smartlook’s user behavior analytics features into your A/B testing workflow. You can get a free 30-day, full-featured Smartlook trial (no credit card required) and start validating the results of your A/B tests today.

Our tool can be used on various types of websites, including those built with popular CMSs like WordPress, Joomla, and Shopify. Implementing Smartlook on your site is as easy as pasting a code snippet (directly or via Google Tag Manager). 

Smartlook also has SDKs for 10 native mobile app platforms, frameworks, and engines — including iOS, Android, React Native, Flutter, Unity, and Unreal Engine. We have detailed documentation to guide you through mobile app implementation step-by-step. If you get stuck or have any questions at all, our Support team is ready and waiting.

Lastly, besides A/B testing tools, Smartlook is also compatible with tons of other solutions that product, UX, and digital marketing teams often use like CRMs (e.g., Salesforce), analytics tools (e.g., Google Analytics and Segment), and much more. 

For an in-depth Smartlook presentation that’s tailored to your business, schedule a demo with our team.

Martin Bolf

is the product manager at Smartlook. Martin is enthusiastic about delivering the best possible customer experience. Prior to joining Smartlook as a product manager, he used to work as a consultant for Oracle NetSuite. Martin has a deep professional interest in biometric signing and work digitalization. He is also an NFL enthusiast and likes to enjoy good food (ideally while watching NFL).