AB vs. Multivariate Testing – Conversion Wednesday



In my last daily concept video, I discussed clickmaps being a good tool to identify parts of your website that aren’t performing. Now that we’ve identified an area to improve, how do we test our ideas to see which one works the best?

There are two types of tests: A/B and Multivariate.

An A/B test tests one variable.  So, let’s say your clickmap shows you that not enough people are clicking on your “keep shopping” button on your cart page. Let’s also say you feel this is important to increase the average order value.

Start with an A/B test. Pick a variable: size, button wording, color, etc. Since it appears nobody sees the button, lets start with size.

Now, just because the name implies “A/B,” it doesn’t mean that it has to simply be just the control and one other version. We can make as many different sizes we want to test as long as size is the only thing that changes.

—- Help us help you! Please take 3 minutes to complete this confidential survey. —

We’ll start with: small (control), medium, and large. We now have three combinations to run on our site: our control and two larger sizes. We’ll run this test with these three versions and see which size performs the best on the site.

But, what if we want to test other factors as well? Such as the color and the wording the button…? Can we do it at the same time?

Here’s where you would use a multivariate test.

Multivariate tests can watch multiple variables at the same time. So, back to the “keep shopping” button example:

Lets say we have the size set up: small, medium, and large. Now when we add the color and wording changes…..we will test the three different sizes, the three different colors, and the three different phrases we want in the button. Our multivariate test will test every combination of these variables to see which ones work the best together. All at the same time!

As you can see, A/B testing is the same thing as a multivariate test, it just contains one variable. Multivariate testing is like running multiple A/B tests at the same time.

You might be wondering, “if an A/B test is the same as a multivariate test, why would we not just use multivariate?” This depends on the traffic to the page you want to test. If you draw up all of the different combinations from this multivariate example, you’ll see that you need to test 27 different variations!

This means you have to divide the total traffic to that page by 27 and wait for all 27 versions to have enough traffic to decide a winner. This can take a long time.

So, first ask yourself: Do I have enough site traffic to run the test? If you do, go ahead and run a full multivariate test. If you don’t, break the test up into a series of A/B tests. It might take longer, but you’ll know for sure that the changes you are testing will have a positive result on your site.

Exclusive Concepts offers a full service testing option for our clients called Conversion Booster that will do the work for you in gathering these analytics and determining the best changes for your site to maximize conversion rate. Then test those changes in an AB or Multivariate test. Give us a call at 1-800-504-4324 if you’re interested in learning more.

We would be honored to perform a free conversion audit on your qualified website today.