How to A/B Test with Google Tag Manager

By | November 22, 2019


– [Julian] In this video, I’m going to show
you can use Google Tag Manager to conduct simple A/B Tests and measure the results within
Google Analytics. All the more, coming up right after this. Okay, today our journey starts at this next
Webpost by Simon Vreeman. Together with Martijn Scheijbeler they developed
an A/B testing solution for The Next Web in order to run A/B Tests through Google Tag
Manager on their website. Now, this is a really large tag blog and it
makes sense to utilize the built in capabilities of Google Tag Manager to do A/B testing for
free. So if you want to find out more about that
solution and how to use The Next Web, check out this post. They were also generous enough to write up
their A/B Testing solution in a GitHub repository where you can find the actual code that we
will implement today. But, let’s start at the beginning. We want to implement an A/B Test on our Demoshop
here. We already have Google Analytics installed
through Google Tag Manager and we now just need to find a test subject. So, I have chosen to… actually look at this
product page. We have a sale button here. What if that sale button would be a different
color? Since we are dealing with Google Tag Manager
and Google Tag Manager is based on JavaScript, you would need to know how to manipulate this
element with the help of JavaScript. So you would need to know a little bit of
JavaScript at least in order to know how to manipulate elements in the dom with the help
of JavaScript. So let’s try this out. How would we pretest our manipulation? We can go into the developer tools which we’ll
find under View. And then JavaScript Console. And we can select this element by right clicking
on it and inspecting this element. And we see in our JavaScript Console, that
we are dealing here with a Span Element with a class of OnSale. Now, we can go over to our JavaScript Console
and select this element with the right commands. And now okay, so it would be “document.getElementsByClassName.” then entering the class name “OnSale” and
we see that we return a correct statement here which selects our element. Now since this is an array, which was the
first element, and now we want to change this style of this element. And specifically, the background… to something
that we can choose, like green. Let’s press Enter here. And we can see the element changes. And this is the command we will use in order
to execute our A/B Test. Now the actual logic when to change the color
will be done by an A/B testing script and we will use that script by the guys of The
Next Web. So let’s implement our A/B Test through Google
Tag Manager. Scroll over to Google Tag Manager. The first step would be to actually activate
a built-in variable. Let’s go over to variables, and configure
our built-in variable and look for the Random Number Variable. This is a variable that will randomly pick
a number and we can use the variable in our A/B Testing script because then Google Tag
Manager will decide randomly which version of the A/B Test to user will see. So, once we have activated that, we can go
over to tags and implement our A/B Testing Tag. Let’s give this all a name. This is a custom tag and it will execute our
A/B Test on the product pages. Now as a tag configuration, we’ll go with
our custom HTML tag, and we can implement the code from the code repository here. We’ll just go to Raw and then mark this all,
copy it, and before we paste it into our custom HTML tag, we actually need to surround it
by script tags. And then we can paste it in… and we now
have a fully built testing solution. What will this actually do? Well first of all, it will execute our A/B
Test, it will also set a cookie so if the user has already seen one or the other version,
he will continue seeing that for the duration of the test. You can also set an expiry date and whatever
version the user sees, will be pushed into the data layer. So there, the data layer pushes within. Now all we need to do is implement our execution
of what we want to happen once a variation should be shown. So let’s go down here. First of all, we have here prefix, this is
for the data layer. We can change it over. In this case, it’s The Next Web. Let’s call this Demoshop. You see here that it uses our random number
variable and then we have here our changes that we can implement. Now, you can do multi-variable testing. You can also test multiple variants at the
same time, but we just want to do a simple A/B Test so we’ll delete our second variables
here. And just in this column, we need to input
our function that we want to execute once the variant should be shown. So we already prepared this, so let’s go back. We have here our code. We can copy again. Put it into our execution function so it will
execute once our variant should be shown. Alright, that’s it. Now we can go ahead and configure our trigger. And we just need to choose the side that we
want to have this firing on. In our case… In our case, we want this just on this Flying
Ninja page for now. So let’s copy just the URL. And input that as a Page View Trigger. As early as possible, only some page views. We’ll go with URL path. Contains our URL here. Let’s give this a proper name. So fire on a Page View and fire on our Flying
Ninja page. Alright, let’s save this. Save our tag and try this all out. Let’s go into the Preview and Debug Mode. Which you can also reach by this button here. And this will put our browser into the right
state. Let’s close our developer tools and we see
our site reloaded and it already fired the first A/B Test because we can see the A/B
Test event here in the data layer. So, let’s look into the data layer. And we can actually see it was something was
pushed here and it actually shows us the version that was shown. In our case it’s zero which means this is
the actual control and not the variant. Now if I reload this page, nothing really
happens and we get the same variables again. That’s because we have a cookie installed
that will ensure that we won’t see multiple, different, other variations and then screw
up our test. So, we’ll be able again to be able to access
our developer tools in order to get rid of that cookie, just to test it out. Under Applications, we have our cookies here,
and we see a cookie is stored with this ds-cookie or whatever you have as a prefix that you
can delete. Just to see if it works. Let’s reload this again. Unfortunately, we’re getting again. Apparently the, yeah, control version. Let’s delete this again. Try it out again. No, no luck this time. This can take a while. And here we go. We see that our code executed and this time
we got the green button. Now if you reload this again, you can also
see the big downside because Google Tag Manager is actually loading asynchronously so you
actually see a little bit of a flash effect once you reload this page. But our A/B Test works on that basis and if
you’re okay with these changes that are made, we’ll be able now to analyze the results. Now, all we need to do is capture data and
analyze our results and this can be done with the help of Google Analytics. So in Google Tag Manager, you already saw
that there was an event pushed into the data layer and we just need to pull out the data
of the right variant and transfer that data to Google Analytics in order to later then
see which users converted based on our changes. So how do we do that? First of all, we need to prepare a data layer
variable to pull out this value. So let’s go over to Google Tag Manager, under
Variables we can build a new user defined variable. This will be our data layer variable of type
event action. We’ll go with the variable configuration here
and choose our data layer variable and enter the key that is here Event Action. Let’s save these. By the way, you could also recreate the event
category and the event label. I will leave that out because these will static
for us. But, if you want to fill them dynamically,
then you could be recreating those as well. Now, all we need to do is send out our data
to Google Analytics and we’ll do this by a Google Analytics Event Tag. Let’s take as a tag configuration, we’ll choose
Universal Analytics. I have already a tracking ID saved in a constant
variable. So, we will use that, and as a Track Type,
we’ll choose Event. Now we can utilize our event variables. For the category, I haven’t created one, so
I’ll just hard code it with A/B Test. Then, the action should be dynamically filled
with the variant that was shown. So we can choose event Action and if you don’t
want this to affect our bounce rate, then we change the Non-Interaction Hit to True. Now, if you want to be more sophisticated
with your A/B Test, you can even set a custom dimension to make sure that your test results
get stored on a user basis in a custom dimension. But, we won’t do this for now. We’ll go ahead and build a trigger and this
trigger will be based on the custom event that is sent to the data layer which is this
A/B Test. So, this will be a custom event and we’ll
file under A/B Test. Let’s configure this. Choose our Custom Event here and as the Event
Name we’ll go with our A/B Test. This needs to model exactly what it says here
in the event key. Let’s save this. Save our Google Analytics tag and refresh
our preview in demo mode. Let’s go back to our page. Refresh it there as well. And you see that we’re getting the first variation
here. Our A/B Test data layer event fired and based
on that, our Google Analytics event tag fired. And transferred that data over to Google Analytics. So, we should be able to see that also in
Google Analytics itself in the Real-Time reporting. We can go here to Events and there was just
an event that happened. Let’s go over to last events, because this
is a non-interactive event. And we see our A/B Test with the Event Action
ds-1-1. So we have here, our variation number one. So, anytime somebody sees a different variation,
this would fire an event. Now what can we do with this data? Within Google Analytics you can obviously
look at how many people saw different variations variations under the Behavior Report. So, this is the different variations that
we saw. And now you can match that up with actual
conversion data if you have for example an e-commerce shop. It would be good to capture conversions on
a purchase basis. You could also build in a goal for a button
click. And that’s what we have done previously. So, we have defined our goals correctly and
you can build a custom report. That shows you your A/B Test results based
on how many people saw your version. And then, how many people clicked on your
Add to Cart or purchased a product. Now this report is not yet filled because
it takes a while to populate once you have sent data in. Obviously, you would also need to make sure
that your test is actually significant and see which variation actually has won. There are multiple calculators out there that
can do this for you. You can enter your visitors to each version,
the overall conversions, you’ll get conversion rates, and your results that will tell you
how much better the conversion was, but also the significance level that lets you know
whether your A/B Test is actually valid. Don’t forget at the end, if you want to publish
this to your users, then you can go ahead and publish a version in Google Tag Manager. Give this hole a name. And your test will be live for your users. So there you have it. This is how you can build a simple A/B Test
with the help of Google Tag Manager. Really, a great way to A/B Test your site
in an affordable manner with Google Tag Manager and Google Analytics. If you want to find out more, check out the
resources in the description below and if you like this video, please give us a thumbs
up and subscribe to our channel because we’ll bring you more of these tutorials every week. My name is Julian. ‘Til next time.

22 thoughts on “How to A/B Test with Google Tag Manager

  1. Alan Reyes Post author

    I see you are pretty popular….How do you know what people want to learn? Thanks!

    Reply
  2. Craig O Post author

    I was just wondering why would you use google tag manager to do this rather than use Content Experiments?

    Reply
  3. שיר גנץ Post author

    I wonder if you can do a test of a whole page using GA experiments.
    for some reason I can't validate the code, and then when the I still activate the experiment, sometimes I get blank page

    Reply
  4. Platon Borodatyi Post author

    Hi there 🙂
    Thanks for your lessons, they are really useful!
    Could you advise me pls, while I'm trying to set the "style" parameter, I receive the error 1.66:
    "VM560:1 Uncaught TypeError: Cannot read property 'style' of undefined"

    What does it mean?
    Thanks.

    Reply
  5. Xavi Post author

    Hello, I followed your instructions and I got my Analytics report with 4 differents event actions. However I am not able to relate those actions with goal clicks. My AB Testing aim was to check which navigation string gets more clicks. I changed the button string using your GTM snipppet and created 4 different options. I also created a GTM event triggered when visitors click in that button (goal completion). My GA custom reports shows my 4 AB testing sessions, and one column with the Goal completions, but it shows 0 goals in all actions. If I check GA Comversions > Goals section I see that there were 1200 completion goals. The problem is how to relate my AB event actions with my Goal completions in the same report. Thank you.

    Reply
  6. Ravi Kumar Post author

    First of all an Amazing video tutorial on A/B testing bro.

    My Query :
    1. We have developed a CMS like wordpress where we can create pages having some content in it with unique slugs i.e., DOMAIN.COM/about-us for instance. Lets say I have created one more page with different content but with slug aboutus (not about-us).
    2. I want to do experiment with these 2 pages using tag manager since I already got tag manager installed in my vertical.

    So my query would be this : every time user hits about-us page, the content should be served either from original page i.e., 'about-us' or from 'aboutus'. How would I do that ?

    Thanks in advance,

    – Ravi

    Reply
  7. Sasho Lazeski Post author

    Hey Julian, thanks a lot for this! I was able to establish the a/b test successfully. However I have a problem measuring the success.
    I measure my user registrations with a destination goal in google analytics and it works. However, when I include event action as a dimension the registrations goal does not show any numbers (zeros). Isn't it possible to use events in conjunction with destination goals?
    Any advice would be appreciated!
    Thanks

    Reply
  8. Kevin Surmont Post author

    Hi Julian
    Thanks for this very helpful tutorial!
    I ran into the following problem; My tag fires but mij event where I can check the data layer isn't showing up.
    I was working on this yesterday too and for some reason it worked. So my event showed up and i was able to check the data layer and continue with this tutorial video.

    Do you have any idea what might cause this? I assume it has to do with the JS code where de datalayer.push happens?

    Thanks alot in advance!

    Reply
  9. Nancy Jain Post author

    This is what you have explained for 1 page. How can do this test across all the product pages?

    Reply
  10. Yasi Liu Post author

    hey!like your videos, simple, delight to learn. Got one question, why not use GA but GTM to do A/B Testing?

    Reply
  11. Sergej Filippov Post author

    Hi! Thank you for your understandable lessons. I've just subscribed to your Masters.
    I have implemented this lesson but abTest sometimes works, sometimes doesn't. Custom abtest script fires but abTest event doesn't appear in GTM debugger. Here is the screenshot. https://goo.gl/3zaZfR
    I guess the reason is because I clear not only tnw-cookie but _ga,_gat,_gid as well, I clear all site's cookies.
    When I delete only tnw-cookie it works. Also it works when I set a trigger All PV with window loaded option. But in such a case variation delay is huge.
    Could you clarify please how to make it work in case all cookies are cleared? And why it happens?

    Reply
  12. Akash shelke Post author

    Hi! Thank you for your understandable lessons. this very helpful tutorial to me ….

    Reply
  13. Deividas Post author

    Hi in console (Summary) abTest not showing up. As in google analytics Events not working as well eventAction empty

    Reply
  14. Aira Lingat Post author

    Hi, how did you do the click to action on the custom reports?

    Reply
  15. The String Guy Post author

    this is obsoleted, Google Optimize is in place

    Reply
  16. Felipe Fernandes Post author

    How can I set up 50/50% (testing/production) using this method?

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *