iOS A/B Testing with Firebase

A/B testing is very helpful technique when applying hypothesis driven development and data driven approaches.

The simple idea of A/B testing is to create a new version (variant B) of a product or a part of a product and deliver this version to a part of the audience. By comparing the conversion of the two variants we can statistically identify the better one.

Firebase is a platform which offers a lot of features and services for native mobile Apps for free. While Firebase does not have an A/B testing feature per se, a combination of two of it’s features namely Analytics and RemoteConfig, makes Firebase a powerful A/B Testing tool for mobile applications.

In this post we will create a very simple iOS application to demonstrate how to use Firebase to implement A/B testing in iOS.
Our application contains only two screens. The first one contains a “Click Me” button, clicking on the button opens the second screen which is our target screen. We will test an orange version of the button (B) against the default blue one (A).

The iOS project contains mainly a storyboard and two UIViewControllers. ButtonViewController.swift manages the first screen with the button and TargetViewController.swift manages the target screen and contains only a simple label. The sample iOS project can be downloaded under

Note: The iOS project contains some more details on how to load the remote config and how to make sure that the config is loaded when we start the test

To learn how to configure the iOS to use Firebase Analytics and RemoteConfig please visit the following tutorials: and

Implementing the A/B test using Firebase in our App is pretty easy and requires 4 steps:

Step 1: Create remote config with condition

We will use RemoteConfig to split the users. In our test 50% of the users should get the orange button.
In the Firebase Console we go to RemoteConfig and click “ADD PARAMETER”. We give the parameter the name niceButton and set the default value to blueButton. Then we create a condition for the parameter by clicking on “Add value for condition”. The the condition is defined so that  50% of the users get a different value for the parameter namely orangeButton.

After saving the condition, the parameter definition would look like this

In the iOS App we use the RemoteConfig to control the color of our nice button. We do this in the ViewDidLoad function of the ButtonViewController

@IBOutlet weak var Button: UIButton!
override func viewDidLoad() {

    let decision = RemoteConfig.remoteConfig()["niceButton"].stringValue!
    if(decision == "orangeButton"){
        Button.backgroundColor =
        Button.backgroundColor =

Note: You need to remove the app from the device or simulator and install it again to be able to see both variants of the button.

Step 2: Create User Property

In order to tell Firebase Analytics which variant of the button the user has, we use the user property feature of the Analytics.
In the Firebase Console we go to Analytics and click “USER PROPERTIES” then click on “NEW USER PROPERTY”. We give the property the name experimentGroup.

In the App, we call the setUserProperty function to set the value of our user property to the selected variant. We do this in the ViewDidLoad function of the ButtonViewController

Analytics.setUserProperty(decision, forName: "experimentGroup")

Step 3: Track the relevant events

In our sample we need to track two events. When the user sees the button and when the user clicks the button and navigates to the target screen.

In the ViewDidLoad function of the ButtonViewController we add the following line:

Analytics.logEvent("buttonShown", parameters: nil)

In the ViewDidLoad function of the TargetViewController we also add the tracking call:

override func viewDidLoad() {
    Analytics.logEvent("targetScreenShown", parameters: nil)

After using the App and clicking the button for several times we can see the events in the Firebase Console after a couple of hours.

Step 4: Create a funnel for the test

The last step is creating a funnel in Firebase Analytics that contains both event buttonShown and targetScreenShown.

In the Firebase Console we go to Analytics and then to “FUNNELS” and click “NEW FUNNEL”. We give the funnel the name buttonTest and choose buttonShown as first event and targetScreenShown as second one.

After creating the funnel we can click on it and get the following view:

HURRAAA we are done and now let’s see how to evaluate our A/B test.

Note: make sure you select a date that make sense to show the data in the funnel

Evaluate the test

Now, we come to the exciting part of the story. We stay on the funnel view from last section and add a filter to our funnel.
Click on Add Filter -> User Property -> experimentGroup -> blueButton

The funnel screen will automatically update and shows that 44.4% of the users who saw the blue button got to the target page.

We do the same thing with the orange button and see that 64.7% of the users who saw the orange button also got the target page.

Our A/B test shows that the orange button is the winner and we know now which variant of the button we want to keep.

Multivariate Testing

In this example we learned how to execute an experiment with two variants using Firebase. We can also use the same methodology to test multiple variants of the button against each other. The only change we need to apply is to adjust the remote config by adding more conditions with different values to it.

Leave a Reply

Your email address will not be published. Required fields are marked *