What the World’s Second Largest Retailer A/B Tests on Mobile
Tesco, the second largest retailer in the world, is a British multinational supermarket that sells groceries and general merchandise. In addition to its brick-and-mortar locations, shoppers can purchase Tesco products on the company’s UK mobile grocery app, from the comfort of their own homes.
Tesco wanted to improve the number of conversions coming from the product details page within its app.
Tesco is constantly striving to simplify the shopping journey for its customers. Due to the amount of feature details customers want to see, and the legally required information, the product details section became cumbersome. The long page meant the static “add to cart” button was pushed far down the page. As a result, it was not clear to customers how to purchase items.
To keep things simple for its customers, Tesco introduced a persistent “add to cart” button that was always present at the bottom of the screen. Tesco’s hypothesis was that no matter how much users scrolled, the ability to purchase an item would always be within sight.
To validate its hypothesis, Tesco created a mobile A/B test with the following “add to cart” buttons:
- A static button (the control)
- A new locked button (the variant)
From this first test, items added to the cart decreased by a disappointing 19 percent. Analyzing the results, Tesco wondered if the locked button wasn’t obvious enough at the bottom of the screen. As the locked button was shown as the default option upon the initial page load, users possibly reacted with “banner blindness” when they didn’t see the button in its usual location.
Tesco reinstituted the original static button and set up a second test. The new test contained two “add to cart” buttons, combined into one experience:
- The original static button (the control)
- The static button above, plus a scrolling button that appeared once a user began to move down the page
Tesco hoped that displaying a static button upon the initial page load, and locking the button once a user started to scroll, would increase the instances of users adding to cart from the product details page. They theorized this hybrid approach would deliver the best of both worlds.
The new test proved successful. Tesco saw a 3.3 percent increase from the original data in items added to cart from the product details page. Tesco rolled out this new version to 100 percent of its iPhone app users. Now that Tesco has optimized the placement of the button, it’s testing the CTA verbiage to continue to improve the app’s purchase rate.
In the words of Lee Mullins, Optimization Manager at Tesco UK:
“In order to serve shoppers better, we need to understand and measure whether customers find changes we make to our products helpful. A platform developed by Leanplum has given us this ability to see how customers respond to small and large changes to our apps. The platform’s ease of use, coupled with some excellent support from the Leanplum team, has helped us to increase our rate of A/B testing and has enhanced our learning of customer behavior on apps.”
What This Means for App Managers
App managers can’t always predict what tests will improve engagement or ROI. Testing multiple ideas is the best way to find out what really works. Tesco’s first test, while a great idea, didn’t produce the expected results. It took a couple of iterations on the A/B test before Tesco found the right combination of buttons, simplifying the shopping experience for customers and increasing its conversions.
As leaders in mobile A/B testing, Leanplum recommends continual experimentation to find the right results. What once worked in the past may not yield the same results today, so it’s important to re-test over time to ensure you’re maximizing app performance.
How Leanplum Can Help
Retailers facing a similar challenge can leverage Leanplum’s integrated platform. Our Visual Interface Editor empowers marketers to make on-the-fly UI optimizations, without coding and or App Store resubmissions. Likewise, our flexible Variable Manager enables you to sync, manage, and dynamically deploy personalized mobile content, releasing updates to your mobile app with the push of a button.
On top of that, we offer comprehensive A/B testing and real-time analytics to shed insights into campaign success, and turn data into action. If you’re interested in learning more about our mobile marketing automation solution, reach out. We’d love to chat. Send us a note at firstname.lastname@example.org.
Leanplum is the most complete mobile marketing platform, designed for intelligent action. Our integrated solution delivers meaningful engagement across messaging and the in-app experience. We work with top brands such as Expedia, Tesco, and Lyft.