This Is the In-App Content You Should A/B Test

Note: This article originally appeared as a guest post on Mappfia.

Close your eyes for a moment, and think back to a time in your life when you weren’t actively learning from a new experience.

Think about the first time you learned the Alphabet Song, rode a bike or drove a car. With every fumble in the song, wobbly push of that bike pedal or sudden slam of the breaks in the car, you learned how to improve your next experience. Eventually, you understood enough to read your first book, cycle without training wheels, and pass your drivers test.

That same motivation to learn and grow should apply to your mobile app strategy.

From the moment you launch your app and acquire your first mobile user, you should begin testing and learning from every interaction.

A vs. B

Flickr / Robert Couse-Baker

The laundry list of items you can A/B test in your mobile app is endless. When you think of mobile A/B testing, you probably think about the cliche test of changing button colors or page layouts. The truth is that mobile A/B testing should envelop much more than that, including content that lives outside of your app.

To help, we’ve put together a list of critical moments throughout the user’s lifecycle — from install all the way through the conversion funnel. In the next few posts in this series, we’ll go over them all.

To start, check out these vital opportunities to improve your in-app user experience.

User Onboarding Flow

Three out of every 10 users abandon an app after the first use. If it wasn’t clear before, a user’s first impression of your app is definitely worthwhile to track.

Start by testing the way a user registers for an account and the sequential tutorial.

Social Login

Apps that force users to register for an account with an email address and password lose users this early on in the funnel. Offer an alternative login with social media accounts. If users can sync your app with Facebook, Twitter or Google+, it removes an unnecessary step.

Try A/B testing a social login without and see if more users move down the funnel. Here are three examples taken from UserOnboard, depicting Headspace, Waze, and Instagram’s version of social logins.

user onboarding best practices



Tutorials aren’t just recommended; they should be mandatory for every app.  If a user understands navigation and primary button functions, this increases the chances of them sticking around.

Create a tutorial and be sure to show how many steps remain in the process. That way, users won’t lose interest and uninstall before understand the app’s value prop. A good idea is to test the order of your flow or the overall number of steps to determine how many users make it through the new funnel.

And of course, keep your navigation in-line with Apple’s best practices as much as possible. This will ensure your experience is intuitive.

Conversion Pages

Every app has something it’s trying to sell, even if it’s not physical products up for purchase.

After all, you’re always trying to convert someone. Perhaps you have a retail app selling clothes, a travel app booking vacations or a media app providing breaking news.

With mobile marketing automation, you can A/B test crucial elements of your in-app UI. This helps you determine which content creates the highest conversions. Try:

  • Altering color of a button
  • Changing the text on a page
  • Switching images
  • Hiding certain elements from the view

a/b testing mobile

Leanplum’s Visual UI Editor enables you to change your app’s interface, like button colors, without App Store resubmissions.

These steps ensure your first-time user experience keeps people engaged through the initial session — and beyond.


In our next article, we break down some great examples of mobile messages you can A/B test. If you’ve been struggling to see results from your push notification strategy, this is a can’t-miss post.

See you then.

Leanplum is mobile marketing automation that lets you shape meaningful experiences for your users at scale — driving engagement and results. We work with top brands such as Expedia, Tesco, and StumbleUpon. Discover more at