WWDC21 introduced new features that will enable developers and marketers to conduct A/B testing on an apps’ store listing. Product Page Optimization will provide an avenue to test conversion for organic channels, while Custom Product Pages will allow A/B testing via channels outside of the App Store.

These new features will both be available directly on the iOS App Store through App Store Connect, replacing the role of many third-party testing platforms.

Google Play experiments and Apple’s new testing capabilities

A/B testing has been available on the Google Play Store through Google Play Experiments since May of 2015. Developers on the iOS App Store, however, have historically had to rely on conversion optimization via Apple Search Ads creative sets, using third-party testing, or deploying new builds to compare metrics before and after the change.

While they provide valuable information, these methods of testing have always come with limitations to keep in mind. Testing using Apple Search Ads or a resource outside of the App Store can give some insight, but unlike Google Play Experiments, these only give metrics for a subset of paid traffic to use as a proxy before deploying. Even when deploying fully to the App Store, the variations in traffic and trends across pre-deployment and post-deployment periods must always be considered.

With Apple’s new testing tools, Product Page Optimization and Custom Product Pages, A/B testing can take place live on the App Store, reflecting real user decisions while preserving the default Product Page as a control- and most importantly, at the same time. This will provide a more accurate view of the impact changes make on conversion, helping developers glean actionable metrics and make data driven decisions with comparative ease.

Product page optimization

An invaluable and long-sought-after tool for conducting A/B testing comes in the form of the new Product Page Optimization feature. How it works is simple and straightforward: alongside an app’s default Product Page, developers will now be able to create three additional Product Pages. These three test pages can employ a variety of treatments to creative assets — the app icon, screenshots, and preview videos — to determine which change most drives conversion.

While it may be tempting to test as many variations as possible, as with any iterative test it is highly recommended to limit the number of assets changed to one per test cycle. This will help eliminate variables, making it easier to isolate which singular variation is most successful in improving conversion rate. Each individual test can run up to 90 days, and both during the cycle and upon conclusion the results can be monitored and compared to baseline performance in App Analytics.

While developers previously had to extrapolate conversion potential from Apple Search Ads testing or third party tools, Product Page Optimization will allow developers can to get a real simultaneous look into how changes impact all of their traffic, at the same time.

Custom product pages

A significant obstacle when designing the perfect Product Page for prospective users is that the perfect page can vary wildly depending on the targeted demographic. Taking a one-size fits-all approach to page design and marketing creates missed opportunities to capture additional audiences and refine messaging for existing ones. Custom Product Pages fit in nicely as an elegant solution for reaching every user in a way that appeals to them.

Custom Product Pages will drive external channels to one of 35 Custom Product Pages, chosen by a developer to reflect the demographics of the environment from which the user is directed. These unique pages can feature variations on screenshots, app previews, and promotional texts, allowing developers to position their app for the intended audience. Each page will have its own unique URL, which can be placed in relevant advertisements or third-party referrals.

For example, let’s say a streaming app simultaneously ran ads on a news app, a social media page for rock music enthusiasts, and a website featuring scary short stories. With Custom Product Pages, the first ad could link to a page positioned for the app’s news streaming features, the second to a page highlighting music offerings, and the third page could focus on podcasts and audiobooks. Employing a clever ASO strategy designed to optimize the 35 different custom pages available, makes the possibilities for expanding an app’s audience are nearly endless.

Custom Product Pages also offer a way to test your creatives impact on conversion across these different avenues. Available in App Analytics, developers can view actionable performance data on impressions, downloads, conversion rate, retention, and average proceeds. This will provide insight into which product pages are driving conversion for specific demographics, and allow developers to refine their assets to increase overall conversion across their paid channels.

As with Product Page Optimization, Custom Product Pages changes the game for running testing- particularly with paid traffic from sources outside of the App Store. A developer can create a unique campaign for their TikTok traffic, another one for their Instagram traffic, and so on.

With the ability to create up to 35 Custom Product Pages, this concept can be expanded further to create multiple custom pages for different subsets of each traffic source, matching the messaging and creative on each one to a custom link. This will allow developers to create cohesive messaging across every step of the acquisition funnel from external channels, run conversion tests and see what performs best.

While this process was once something that required support from a third party tool, Apple’s new process effectively brings what has been in place for Apple Search Ads creative set testing to other UA channels. Custom pages can be set up, developers and marketers can distribute the URLs to the appropriate campaigns on the appropriate platforms, and measure performance metrics via App Analytics.

Overall

A/B testing on the iOS App Store has historically come with extraneous variables to consider and additional cost to support. Once Product Page Optimization and Custom Product Pages are released, developers will have long-awaited powerful tools to get actionable conversion data and fine-tune external marketing campaigns, directly from App Store Connect.

While these new features will totally revolutionize A/B testing on iOS, it is important to enter into this process wholly prepared. Continued testing with currently available methodologies as part of a solid App Store Optimization strategy is critical to being prepared for the changing testing landscape of the App Store the moment it arrives.