A/B Testing

What Is A/B Testing?

A/B testing is a type of test in which users are separated into groups (variants) and each group is served a different content experience. The control group is served the default experience while the variant is served the alternate experience. The variant is then compared against the control with metrics/goals to establish the winner.

How Does Mobile A/B Testing Work?

Mobile A/B testing can either test a single variable, for example the position of a CTA button, or multiple variables at the same time. The latter is known as multivariate testing. Marketers should be cautious when testing multiple variables because these tests take longer to reach statistically significant conclusions.

A/B testing can be used not only to improve customer conversion and retention rates, but also to ensure that any changes made to an app have not made it worse. Mobile A/B testing can be used to test both in-app and external variables, from buttons and images to emails and push notifications.

Campaigns are set up so that two different audiences are served different versions of these variables at the same time for a specific time period. Using an intuitive A/B testing dashboard, marketers can see the most significant results at a glance. The group that achieves the most goals will determine what changes will be made to the app in the future.

For a successful mobile A/B testing campaign, marketers should consider:

  • Testing everything, from message campaigns to user interfaces. Leanplum‰’s solution allows for comprehensive testing without code changes.
  • Using multiple variables. There are no limits to how many different variants of a variable you can try, so marketers should try many different scenarios to gain sufficient user data.
  • Segmentation and targeting. For each campaign, marketers should define specific audience groups based on events or behaviors.
  • Tracking metrics. Analytics are the most valuable part of an A/B testing campaign. Leanplum makes analytics easy by surfacing the most significant changes, like a 47 percent hike in average daily user revenue, in the dashboard.

What Does Mobile A/B Testing Mean for Marketers?

Mobile A/B testing means that marketers can make changes to their apps and campaigns with confidence. Update your app without fear of losing customers to poor user experiences. Even the most superficial of changes can make a huge difference to ROI.

In a broad sense, marketers benefit from a deeper knowledge of their customers and their behaviors. Leanplum‰’s testing system can track a huge variety of in-app and external variables, helping to create optimized, relevant campaigns.

For more information, read our 6 Minute Guide to Mobile A/B Testing.