Guide to A/B testing with LiveSession

Guide to A/B testing with LiveSession

Find out what and find out why

A/B testing is no longer a niche optimization technique. Did you know that 77% of companies run A/B tests on their websites?

Despite the volume, only 1 in 8 A/B tests results in significant changes for the business. In this guide, we’re going to show you how you can make your test more insightful with LiveSession.

Let’s dive in:

Qualitative data in A/B testing

Qualitative data is not an alternative to quantitative findings. It’s actually quite the contrary! These two methods are supposed to complement each other and help you gain a deeper understanding of user behavior.

While numbers will show you what is happening, qualitative insights will help you find the reasons why it’s happening.

For instance, you might have a landing page with quite a lot of traffic but unsatisfying results. While many people visit it, the conversion rates and bounce rates may not meet your expectations. Now, what should you do about it?

Instead of trying to take a shot in the dark, you can turn to qualitative data and see how the users interact with your landing page. Perhaps a button is not working, or they just can’t find what they’re looking for?

There is a method that is sure to help you out:

Get more context with session replays

You can easily access this kind of information through session replays. This qualitative user research method has a range of od advantages:

  • It’s affordable – Conducting real-time interviews with your users comes with costs that can be hard to cover for some. Session replays, on the other hand, are much more budget-friendly. All you need to do is add a piece of code on your website and you’re ready to start.
  • It happens in the background – Session replays don’t need any involvement from your side. They’re recording in the background and the material is ready for your analysis.
  • It fights the social desirability bias – When faced with a researcher, the users tend to respond in a way that seems more desirable. In the case of session recordings, the users are not aware they’re being watched. Because of this, their behavior is more natural and not affected by this phenomenon, known as the social desirability bias.

Use qualitative research to design A/B tests

Qualitative data can suggest you potential solutions to the problem. When you watch how the visitors use your website, you start to imagine what can be done to solve the issue. Below we’re describing our own case study so you can see exactly how it works!

How LiveSession can help with A/B testing

LiveSession is a session recording and qualitative analytics platform. It can also be connected with a range of other tools, including Google Optimize, one of the most popular solutions for A/B testing.

When you connect your Google Optimize A/B test to LiveSession, you will be able to see sessions that illustrate how users interacted with different versions of the test. In other words, it’s a fast and easy way to enrich quantitative data with qualitative data.

Below you’ll find a tutorial on how to connect LiveSession and Google Optimize:

How to connect LiveSession with Google Optimize

We’ll start with an example so you can easily see how it works.

Let’s say that you want to measure how the button color affects conversion and bounce rate.

In our example, we’re going to test two different colors: green and red. Originally, the “Purchase Now” button on our site was green:

Guide to AB testing - 1

The best way to find out is to conduct an A/B test with different color variants. Here’s where Google Optimize comes in:

It’s really simple: just go to https://optimize.google.com/ and create your first experiment:

Guide to AB testing - 2

Define the name (1), the URL of the website you’ll be testing (2), and click Create (3):

Guide to AB testing - 3

Next, go to the Targeting and variants section. Click Add variant (1), define the name of the variant (2), and click Done (3):

Guide to AB testing - 4

Click Edit to set a variant directly on the website:

Guide to AB testing - 5

TIP: The easiest way to create a variant is with the official Google Optimize extension.
We’re going to use it in this tutorial.

Guide to AB testing - 6

Select the button (1). Find the background-color property (2), change it to red (3) and click Save (4).

In the next section, connect (1) your Google Analytics account and choose (2) a property for the Google Optimize Container. Remember that it has to be the same property that you’re using on your website.

Guide to AB testing - 7

Create an objective to measure:

Guide to AB testing - 8

It this example, we’re going to measure Bounces:

Guide to AB testing - 9

Go back to the top and click Start:

Guide to AB testing - 10

Click View instructions to get information about your Container ID:

Guide to AB testing - 11

Copy the selected ID:

Guide to AB testing - 12

Next, go to Google Tag Manager (https://tagmanager.google.com/), chose a container for the same website, and open the Tags section:

Guide to AB testing - 13

Click New to add a new tag:

Guide to AB testing - 14

Find Google Optimize on the list:

Guide to AB testing - 15

Paste your container ID and connect your Google Analytics settings (or create new ones) with the same property you’ve connected with Google Optimizer. Click Save.

Guide to AB testing - 16

Go back to Google Optimize and copy the Experiment ID from the Measurement and objective section:

Guide to AB testing - 17

In Google Tag Manager, add another Tag with a Custom HTML.

Guide to AB testing - 18

Paste this code:

<script>
  function gtag() {dataLayer.push(arguments)}
  gtag('event', 'optimize.callback', {
    name: '<EXPERIMENT_ID>',
    callback: function(variant) {
      __ls("setCustomParams", {params: {ab_variant: variant, ab_id: "<EXPERIMENT_ID>"}});
    }
 });
</script>

Paste the copied experiment ID in the <EXPERIMENT_ID> field. This is a connector between LiveSession and Google Optimize.

Guide to AB testing - 19

Click Save, and then Submit...

Guide to AB testing - 20

...and Publish the changes:

Guide to AB testing - 21

In Google Optimize, click Start to start testing!

From now on, your website visitors will see the two different versions of the Purchase Now button.

Go to LiveSession, and choose the Custom param filter:

Guide to AB testing - 22

Set the ab_id with the Experiment ID value. Then, set the ab_variant. In our case, 1 represents the first variant with the red button.

Guide to AB testing - 23

Save the results as a segment (1). It will appear in the segments section (2). Click at the ab_variant filter (3):

Guide to AB testing - 24

Change the original version’s ab_variant to 0 (1), apply filters (2), and click Save as a segment.

Guide to AB testing - 25

Use the created segments to compare two groups of users. With LiveSession, you can precisely check how the behavior varies between different versions. You can always preview the properties to see which variant the user has interacted with.

What’s more, you can also see the results for the segment in Google Analytics. Just create a segment in Google Optimize:

Guide to AB testing - 26

Here’s what it looks like in Google Analytics:

Guide to AB testing - 27

Case study: how to analyze test results with LiveSession and Google Optimize

To show you how our integration works in action, we’ve decided to use a real-life example!

We’ve set up a little A/B test on our home page. We’ve created two different versions of the hero section on our homepage. Each one of them was shown to half of the website’s visitors. The 0 variant had only one column, while the 1 version had two:

Guide to AB testing - 28Guide to AB testing - 29

The what: Google Optimize test results

Here are our results from Google Optimize:

Guide to AB testing - 30

As you can see, the one-column version performed significantly better than the two-column one. The conversion (from visit to signup) was twice as high for the original version.

What’s more, the Optimize analysis comes with actionable statistical data. The Probability to be best section shows you how likely it is that the winning variant really is the best one. You can also see the range of conversion rates you can expect for both cases.

That’s it for the results, but here comes another question – how exactly do people interact with these websites? Why is one version better than the other?

LiveSession can help you find the answers.

The why: LiveSession recordings

The recordings of users who took part in the A/B test are saved as a separate segment:

Guide to AB testing - 31

It’s easy to say that with session recordings you can see how A/B test participants interacted with the website.

It’s harder, however, to imagine what exactly you can see in these recordings. We don’t want to leave it to your imagination only. Have a look at some of our findings from the described A/B test:

  • Users tend to click on things that are not clickable, such as the “play” button on this illustration:Guide to AB testing - 32
  • You can see where people tend to stop when they’re browsing the page. For instance, they spend more time reading the section about data privacy:Guide to AB testing - 33
  • You can track the most common next steps, just like you’re sitting next to the user! In most cases, the visitors go to Features and Pricing after seeing the main page:Guide to AB testing - 34

What’s next?

We hope this guide helped you understand how you can get the most out of A/B tests with LiveSession!

Want to read more? You might also be interested in these resources:

5 Tips for Better A/B Tests

How to Analyze Session Recordings (Effectively)

5 Easy Ways to Increase Landing Page Conversion Rate

Want more knowledge?

Get more tips and insights on UX, research and CRO. Zero spam. Straight to your inbox.