Simple Split Test With Google Tag Manager

ShareShare on Google+12Tweet about this on Twitter18Share on LinkedIn11Share on Facebook19

Over the last couple of posts I’ve mainly been doing proof-of-concept (POC) tests with Google Tag Manager. The great thing about a POC is that you don’t really need to have any viable results or insight-driving technological innovations. The point is to showcase some feature of the platform on which the experiment was conducted.

In this post, I’ll take a care-free step into the world of POCs again. My goal is to do a simple split test in order to identify which variant of a landing page (or key element thereof) produces the most conversions.

This isn’t a replacement for any of the REAL A/B testing tools out there. Rather, this is a way for me to showcase yet another way to utilize the wonderful complexity of GTM’s JavaScript wizardry.

Anyway, this is what you’ll (hopefully) take with you after reading this post:

  • How to modify a DOM element
  • How to set a cookie
  • How to show a different variation to 50 % of users

Combine these and you have a split test in the making!

The end result

This is how it’s going to go down. Exactly 50% of people viewing the experiment page will see the following button:
Click here to make dollars
This is also the control variant. It’s the one I’ve used for years and years to make millions upon millions to my faithful followers (just kidding). However, I want to see if a more nonchalant approach to click baiting would work (and I want to appeal to the younger, less critical demographic), so this is the test variant shown to the other 50%:
Click here, whatever
So I’ll be measuring just how many clicks each of these buttons collect. In the end, I should see it in my reports both as events and as a Custom Dimension. Here’s what a goal conversion report would look like for the two variations.
Split Test Report in Google Analytics
It’s not exactly hard science, but it got the job done. Surprisingly, the control variant was stronger, producing more conversions (who would’ve guessed?!).

Setting up GTM

For this experiment, you’ll need the following:

  1. 1st Party Cookie Macro {{Split Variation Cookie}}
  2. Random Number Macro {{Random Number}}
  3. Custom HTML Tag
  4. Custom Dimension
  5. Universal Analytics Event Tag

1st Party Cookie Macro

You’ll need a 1st party cookie to make sure the same user sees the same variant every time s/he visits the page. So when the variant is first assigned to the user, this cookie is also written. The cookie will return the value of the variant, which is important for the Event Tag and Custom Dimension as well.

  1. Create a new 1st Party Cookie Macro
  2. Name it Split Variation Cookie
  3. Set Cookie Name to splitVar

So something like this:
Split Variation cookie
This macro will return the value of cookie splitVar which, as we will soon learn, stores the variant the user should see during the experiment.

Random Number Macro

I’ll use the Random Number Macro to sample 50 % of my visitors. It’s a nice trick, and I first saw it on Dan Russell’s blog.

  1. Create a new Random Number Macro
  2. Name it Random Number

Here we go:
Random Number Macro
So every time this macro is called, it returns a value between 0 and 2147483647. Can you see how useful this is for sampling? If I check if {{Random Number}} < 0.5*2147483647, it should return true 50 % of the time.

Custom HTML Tag

This is where the magic happens. You’ll need a Custom HTML Tag to perform a wide variety of things: modify the DOM element, identify the user’s preferred variant, set and read a cookie, etc…

Let’s start with the code itself. This is what the tag should have. Remember to have it fire on all pages on which the experiment is run.

<script>
  function testVariant() {
    controlElement.style.backgroundColor="#000099";
    controlElement.innerHTML="<strong>Click here if you want, whatever</strong>";
    return variantTwo;
  }
  
  var controlElement = document.getElementById("call-to-action"), // Set ID of control element
      variantOne = "clickDollars", // Variation 1 name
      variantTwo = "clickWhatever", // Variation 2 name
      variant = variantOne, 
      randomNumSample = 1073741824;

  if(!{{Split Variation Cookie}} && controlElement) { // If cookie isn't set run code
    if({{Random Number}} < randomNumSample) {
      variant = testVariant(); // For 50 % of hits, fire Variation 2
    }
    var d = new Date(); // Create cookie
    d.setTime(d.getTime()+1000*60*60*24*730);
    var expires = "expires="+d.toGMTString();
    document.cookie = "splitVar="+variant+"; "+expires+"; path=/";
  } else if({{Split Variation Cookie}} == variantTwo && controlElement) { // If user has only seen Variation 2
    variant = testVariant();
  }
</script>

Let’s go over this code.

Lines 2–6 contain the code which modifies a pre-set DOM element. Thus this function operates the actual split test, showing a different variation of the landing page for 50 % of users. This function returns the name of variation 2.

Lines 8–12 set up some variables for this script to work. Change call-to-action in getElementById(“call-to-action”); to match the ID attribute of the HTML element you want to dabble with on the page. Change string values for variationOne and variationTwo to match how you want to name or two variants. These values will be used in your GA reports. Finally, the variable randomNumSample contains a numerical value exactly 50 % of the maximum the Random Number Macro can return.

Lines 14–21 first test if you’ve already been assigned a variation by checking your cookies. If no cookie is found, you’re a new user, and a variation must be assigned to you. In this case, the script randomly assigns you either the control variant (the default, “clickDollars”) or the test variation (“clickWhatever”). Next, it writes this information in a cookie which will stay with you for a long time.

Lines 22–24 are executed if the cookie is found. In this case, if the variant that was assigned to you was the test variation, the DOM altering function from the beginning (see lines 2–6) is run again to make sure you always see the same variation of the landing page. If the variant that was assigned to you was the control variation, nothing happens, since that’s the default version of the landing page.

Some things to note. First of all, the cookie is set for two years, so it mimics the _ga cookie. The point is that the user should always see the same variation (provided they’re using the same browser). For some split tests this isn’t necessary, so you can change the code to forgo the cookie check. Also, all the functions test for undefined values (both of the cookie and the control element), so I didn’t see the need to add unnecessary try…catch blocks or anything else.

Pro tip – Make use of the rarely utilized custom firing schedule for this tag to make sure your experiment only runs a certain time:
Custom firing schedule for tag

Custom Dimension

You’ll need to create a new user-scope Custom Dimension, so that you’ll have one additional segment to play around with in your data.

  1. Go to Google Analytics admin, and choose the property you’ll be tracking these hits to
  2. Choose Custom Definitions > Custom Dimensions, and create a new user-scope CD
  3. Give it a descriptive name (I named mine just Test variation)
  4. Make note of dimension index

So something like this:
Custom Dimension for test variation
You could also use a session-scope or even a hit-scope dimension, depending on the scope of your test. However, even though I’m not a professional CRO practitioner (far from it!), I see the benefit of observing interactions on a multi-visit, user-level rather than the relatively tiny realm of a single visit or hit. However, some web design questions can probably be answered on a more limited scope, I guess.

Universal Analytics Event Tag

The last bit you’ll need is some event that’s firing and sending your data to Google Analytics. Because I’m observing a button (that I’ve encased in an <a href…> element), I’m also using a Link Click Listener. If that says nothing to you, be sure to check out my guide on auto-event tracking.

So anyway, I’ve got a Link Click Listener firing on the test page. What I want to know is just what variant the user was seeing when they clicked the button. This is how I’ll count my conversions and determine which variant was the winner. I’ll also send the Custom Dimension with the event, so that the user is properly annotated with the variant name. I’m sending the CD with the event simply because it’s convenient for the scope of this article. You should definitely send it with the pageview, so that you can segment your visitors properly (e.g. out of people who saw the control variant, how many clicked the button; out of people who saw the test variant, how many clicked the button).

My Event Tag looks like this:
Event Tag for split test
As you can see, I’m using the cookie value (“clickDollars” or “clickWhatever”) to annotate the event and the Custom Dimension! What a nice way of saving time and resources. So when someone clicks on the default variation, for example, the Event Tag will send something like this to GA servers:

Event Category: Call-to-action
Event Action: clickDollars
Event Label: /call-to-action-page/subscribe/
Dimension 1: clickDollars

Remember to change the index number of the dimension to match the CD you created in the previous chapter!

The firing rule I’m using is simply:

{{event}} equals gtm.linkClick
{{element id}} equals call-to-action-link

This rule will thus only fire when a click occurs on an element with ID “call-to-action-link”. Conveniently, this just happens to be my control element.

Conclusions

Well here it is. A simple split test done solely with GTM and some JavaScript magic. Now let me reiterate: this is hardly the best way to do split testing (we’re still waiting for Content Experiments to make their way to GTM + Universal Analytics..) but it does showcase the power and might of a tag management system. I am, after all, directly manipulating an on-page element without touching server-side or front-end code! That’s amazing…and scary.

Before you start fiddling with your DOM (wow, that sounded dirty), make sure you know just what you’re doing and to which element. This is crucial, because you don’t want to accidentally set off a full-scale website redesign without actually, you know, intending to.

Note also that because you’re changing the element in an asynchronously loading tag, it’s more than possible that the user will see the control variant before the script overwrites it with the test variant. This isn’t a good thing, as if the control group realizes they’re participating in an experiment, it will hurt the reliability of the results.

There are a couple of nice things to snag from this to your other scripts. Make use of the set-cookie function I wrote in the Custom HTML tag. It’s a versatile solution, and I’ve actually used it before as well. The random number sampling is simply genius, and I love how easy it is to operate!

ShareShare on Google+12Tweet about this on Twitter18Share on LinkedIn11Share on Facebook19

Comments

  1. Josh says

    Once again you blew my mind with what GTM can do! Love the proof-of-concept stuff. Thanks for writing awesome posts!

  2. says

    An ingenious post Simo!
    Its so simple and sophisticated… really awesome…

    I just wanna ask – why should you check if controlElement is set?
    I understand that you need to prevent undefined situation but if you set the value of controlElement *before* the if statement – how it can happen?

    • says

      Hi Shuki,

      Thanks!

      I don’t check if controlElement is set, I check if it’s defined. Yes, I do assign a reference to controlElement earlier, but on pages without the “call-to-action” id, that reference would not resolve and thus the variable would be of undefined type. If I were to process controlElement without checking for undefined, then on pages where the reference is missing I would run into more errors.

  3. says

    Thanks! This article helped me create a split test for retargeting platforms. I wanted test if Google Remarketing or AdRoll performed better.

    Rather than modify the “controlElement” I triggered a “custom event” macro called “event” using the dataLayer.push. Then create a rule that says, if the “event” macro equals “google” then fire off the Google Remarketing Tag. I repeated the steps to create rule for “adroll”.

    Here is the code (could be simplified):

    function controlVariant() {
    dataLayer.push({‘event’: ‘google’});
    return variantOne;
    }
    function testVariant() {
    dataLayer.push({‘event’: ‘adroll’});
    return variantTwo;
    }

    var randomNumSample = 1073741824,
    variantOne = “google”, // Variation 1 name
    variantTwo = “adroll”, // Variation 2 name
    variant = variantOne;

    if(!{{Split Variation Cookie}} && randomNumSample) { // If cookie isn’t set run code

    if({{Random Number}} = randomNumSample) {
    variant = controlVariant(); // For 50 % of hits, fire Variation 1
    }
    var d = new Date(); // Create cookie
    d.setTime(d.getTime()+1000*60*60*24*730);
    var expires = “expires=”+d.toGMTString();
    document.cookie = “splitVar=”+variant+”; “+expires+”; path=/”;
    }
    else if({{Split Variation Cookie}} == variantOne && randomNumSample) { // If user has only seen Variation 1
    variant = controlVariant();
    }
    else if({{Split Variation Cookie}} == variantTwo && randomNumSample) { // If user has only seen Variation 2
    variant = testVariant();
    }

    • says

      Hi Phillip,

      Excellent idea, thanks for sharing!

      There’s a problem in your code, though. In the line if({{Random Number}} = randomNumSample) {, you need to change the equals sign to either larger or smaller than (< or >). With just the equals sign it’s a value assignment, not a comparison.

      Also, in the two else if constructs at the end, you don’t need && randomNumSample since that variable isn’t referred to in the code that follows.

      • says

        Yes, you are correct. Here is the fixed code:

        function controlVariant() {
        dataLayer.push({'event': 'google'});
        return variantOne;
        }
        function testVariant() {
        dataLayer.push({'event': 'adroll'});
        return variantTwo;
        }

        var randomNumSample = 1073741824,
        variantOne = "google", // Variation 1 name
        variantTwo = "adroll", // Variation 2 name
        variant = variantOne;

        if(!{{Split Variation Cookie}} && randomNumSample) { // If cookie isn't set run code

        if({{Random Number}} = randomNumSample) {
        variant = controlVariant(); // For 50 % of hits, fire Variation 1
        }
        var d = new Date(); // Create cookie
        d.setTime(d.getTime()+1000*60*60*24*730);
        var expires = "expires="+d.toGMTString();
        document.cookie = "splitVar="+variant+"; "+expires+"; path=/";
        }
        else if({{Split Variation Cookie}} == variantOne) { // If user has only seen Variation 1
        variant = controlVariant();
        }
        else if({{Split Variation Cookie}} == variantTwo) { // If user has only seen Variation 2
        variant = testVariant();
        }

      • says

        Hey Philip,

        Actually, there are still some problems:

        if(!{{Split Variation Cookie}} && randomNumSample) { … no need to check for randomNumSample, as it’s declared in the same scope a few lines before.

        if({{Random Number}} = randomNumSample) { … this is still a value assignment, not a comparison. You need to have if({{Random Number}} < randomNumSample or > randomNumSample, not =.

        Also, since controlVariant() return variantOne which is the default value for variant, the assignment needs to be:

        if({{Random Number}} < randomNumSample) { variant = testVariant(); }

        Otherwise the variant will never get assigned to testVariant().

    • says

      Ah, I see now :) You’re right, an unfortunate feature of WP comments…

      So absolutely, your code should work perfectly. Good job and good idea!

  4. Ido Inchal says

    Hey there!

    Very good post thank you :-)

    Once before I used this GTM tactic to create an ABT where I couldn’t implement codes freely and rapidly.

    This is a great work around especially for affiliates, agencies etc.

    Thanks for sharing I’ll make sure to follow your future posts.

    Ido

Leave a Reply

Your email address will not be published. Required fields are marked *