A/B Testing (Beta)

A/B testing makes it easy to run and analyze in-app purchases and subscriptions experiments to find the best product mix and parameters like subscription price, duration, and other parameters that maximize your app's revenue in the long run.
It is an out-of-the-box solution that does not require any third-party service. You can roll out the winning version of the experiment to all users instantly without releasing a new app version.

📘

You need to use Qonversion's Subscription Management mode to run A/B experiments with Qonversion. Check our Quick Start guide to integrate Qonversion.

How does it work?

A/B testing uses Qonversion offerings. An offering is a group of in-app products that you can show to a user on a single screen. For example, you can offer one set of products to user A and another set of products to user B. Offerings allow managing in-app products remotely.

1. Setting up an A/B test

Define two sets of products that you want to test.
For example, you have the following two product sets that you want to test:

Group A:

• annual subscription with a 3-day free trial at 39.9$
• monthly subscription without a trial at 5.99$
• lifetime offer (non-consumable in-app purchase) at 99$

Group B:

• annual subscription with a 7-day free trial at 49.9$
• monthly subscription without a trial at 5.99$
• lifetime offer (non-consumable in-app purchase) at 99$

1. Group a set of products into an offering

Create offering A. This can be any of your existing offerings, you do not need to create a new one. Check the details on creating offerings here.

2. Create the A/B test

Navigate to the A/B Testing dashboard and click Create.

Provide the following details:

• Variant A: the offering you want to test;
• Name: the test name, displayed only in your dashboard;
• Description: test description for your reference;
• Exposure: the percentage of users exposed to this experiment
• Create Variant B: add the second offering for the test group.

Click the Run A/B test button to start the test.

2. Displaying Offerings

You need to display in-app products using an offering ID to run A/B tests. Please check the documentation on displaying the products here.

3. Making Purchases

Make a purchase using Qonversion's product object. Please check the documentation on making purchases here.

4. Monitoring your experiment results

Qonversion measures the following metrics for each experiment:

• Users
• Active Users
• Purchases
• Revenue
• Trial started events
• Trial converted events
• Trial converted revenue
• Subscription started events
• Subscription revenue
• In-app purchase events (non-subscription products)
• In-app purchase revenue (non-subscription products)
• Refund events
• Refund revenue
• Conversion to trial start, %
• Conversion to subscription start, %

Chose the test in the A/B Testing dashboard to review its results.