The Science of Conversions: A Complete Guide to Multivariate Testing

Daniel // Analytics


February 27  

Digital marketing is one of the most lucrative ways that businesses can make money. In fact, not having an online presence will only make it harder to keep up with your competitors.

But it’s not always easy to understand the ins and outs of this process. Even more difficult, is finding the best strategy for you.

You put all that time, money and effort into creating a website and driving traffic to it. You’ve done everything right so far. But if those visitors aren’t converting, then they’re just costing you money. 

Good news is, you’re not at the end of the road, you’re just in the middle of the process. The next move is to maximise the value of each visitor to your website. 

Multivariate testing is your opportunity to fine-tune your digital marketing tactics and target your audience as efficiently as possible. To what end? 

Conversions, conversions, conversions.

Not sure where to start? Don't worry, we’ve got you covered.


Before We Begin... 

Before we go further, it's important to understand what A/B testing is, because multivariate testing is an expanded version of A/B testing.

A/B testing, or split testing, compares two versions of a web page in order to see which one is more effective. For example, you might direct half of your audience to one specific landing page and the other half to a separate landing page.

Once enough traffic has been funneled through both pages, you then compare the conversion rate of each to determine which version was more effective. Assuming that both tests were under the same controls (emails sent at the same time, both groups had an even number of participants, etc.), there should be a relatively low margin of error.

While this practice is a solid digital marketing tactic and can provide noteworthy results, it's not the optimal method for those who are looking to test a large number of elements.

You put all that time, money and effort into creating a website and driving traffic to it. You've done everything right. But if your visitors aren't converting? Then they're just costing you money.  But there's a to boost those conversions.

Click to Tweet

What Is Multivariate Testing?

Multivariate testing is A/B testing on steroids. 

As the name implies, multivariate testing tests multiple variables at once in order to find out which combination of elements is most effective in driving conversions.

When would you use this method? 

You might test a landing page, if you have 3 different headers and 2 different images you’re deciding between. You can test each combination of those elements, by creating a different landing page version for each combo. 

Once you segment your audience, you then funnel each segment into one of the 6 different landing pages. When you have enough data, compare the conversion rates of each page and you’ll know which combination is most effective.

A/B Testing vs Multivariate Testing

There are 2 key differences between A/B testing and multivariate testing. 

A/B testing requires less traffic to get meaningful data, because you only need to split your traffic into two testing groups (A and B). This is why it provides quicker results after you measure your analytics

The multivariate method will likely involve testing more than two different combinations of elements, splitting your audience into 3 or more groups. This will require more traffic and more time to produce meaningful insights. 

Let's take a look at what each method might look like.

A/B Testing

AB testing shown as a group of users split into subsets A and B.

David owns an athletic apparel company and is building a landing page for people to pre-order his latest trainers release. 

He's unsure of which photo to display. His latest trainers come in four different colours, but of his previous releases, white and black were the most popular colour choices. 

He creates two separate landing pages to run an A/B test. Both include a call to action to pre-order the trainers, but Version A displays the white trainers and Version B displays the black trainers. 

He then uses email marketing to drive traffic to both pages, splitting his audience of 10,000 into two 5,000-person segments. After a short period of time, he checks his analytics to find that the landing page with the black trainers (Version B) resulted in more conversions.

Multivariate Testing

Multivariate testing shown as a group of users split into 8 testing subsets..

For his next trainers release, David wants to see which combination of trainers colour and call to action generates the most conversions. He decides to make different landing pages for each of his trainers' colours:

  • Jet (black)
  • Chrome (white/silver)
  • Oasis (tan/sky blue)
  • Autumn (orange/black)

Along with two different CTAs:

  • 'Reserve Your Pair'
  • 'Pre-Order Now'

There are a total of 8 combinations that David needs data for, so he needs to split his audience into 8 separate groups when driving traffic. So, he sends 1,250 people to each version of his pre-order page.

Although the image associated with the CTA is a specific trainers colour, visitors are able to view the other colours before the pre-order. This gives David data on which image/CTA is most effective and not just which pair of trainers is more popular.

Despite past popularity of white and black trainers, the results of this test showed that the landing page with the Autumn coloured trainers and the 'Pre-Order Now' CTA had the highest conversion rate. 

It’s important to note that it would be difficult for David to obtain accurate data from multivariate testing if his audience were significantly smaller (such as less than 1,000 people). But since his audience was large enough, both tests were able to give him actionable insights as to which versions are most effective in boosting conversions. 

The question is, why choose one over the other? 

Running an A/B test requires less traffic and offers quicker results than a multivariate test. So... why is a multivariate test the better option? Click on to find out. 

Click to Tweet

What Are the Benefits of Multivariate Testing?

The main feature of multivariate testing is the source of its main benefit: testing multiple elements at once.

Why does this matter?

Saves You Time

I know, I know. I said that A/B tests are quicker. They are quicker, per individual test. But, it’s not that multivariate testing is slower than A/B testing. It’s that it’s doing a lot more work for you.

Think back to David and his apparel company. The insights he gained from one multivariate test would have required 8 separate A/B tests.

Gives You More Data

A multivariate test gives you far more data than an A/B test. In fact, a handful of A/B tests may still provide less data than a single multivariate test.

That’s why it takes a bit more time per single test. It’s giving you more information to work with. 

But it doesn’t just give you more data...

Actionable Insights

...it gives you better quality data that leads you to actionable insights. Think of it like a spotlight showing you where and how you should take action next.

These insights will give you more clarity about your target audience. Things like, what colours they prefer or what style of copy they respond best to. 

That’s incredibly valuable information, because you can use it now and in the future to better fine-tune your campaigns. It will allow you to create new pages or optimise existing ones for the best conversion rates possible.

It's not that multivariate tests are slower than A/B tests. It's that they're doing a lot more work for you. Read on to find out why they're worth the time... 

Click to Tweet

How To Do Your First Multivariate Test in 4 Steps

Multivariate testing is a hugely powerful form of research. It might seem complicated at first, but once you get past the jargon, it’s a method that any entrepreneur can and should take advantage of to boost conversion rates.

Luckily, it’s relatively easy to set up. I’ll walk you through the steps. 

1 - Define Your Goal

Before you do anything, answer this: What does a conversion mean to you? 

For some, it's making a sale. For others, it's getting visitors to sign up for an email newsletter. 

Or maybe you'd like people to download a free eBook on your site. After you get their email to send them the document, you'll be able to send them offers and promotions in the future.

Make sure you know exactly what conversion means to you. It’s what you’ll be measuring for.

2 - Develop Page Variations

These will vary from business to business, but common variations for multivariate testing pages include images, lines of copy and calls to action.

Remember, one multivariate test can accomplish a lot for you, because it can handle a lot. So come up with at least 5 or 6 possible versions of your landing page to test.

3 - Segment Your Audience + Drive Traffic

Once the versions are set, you'll need to segment your audience and funnel them to your various landing pages. The method you use to drive this traffic will vary, but often includes social media, email campaigns, paid ads, etc.

Top Tip

More traffic means more reliable. The larger the audience for each page version, the more reliable the conversion rate data.

For example, if 40 out of 100 people make a conversion on a page, you'd have a 40% conversion rate. That's a strong number. But if it was 4,000 out of 10,000 users making a conversion? That would be the same 40% conversion rate, but at a much larger scale.  Larger audience = better data. 

4 - Analyse Your Metrics

Once your testing is complete, you'll be able to take a look at your results and determine what content was most effective in leading to conversions.

But, don’t stop there! 

It’s a good idea to take your results and apply them to a similar multivariate test in the near future. Focus on testing new variables. If you looked at images and CTA last time, you could use that data to create a new test that compares how different form fields, buttons, etc. affect conversion rate.

One multivariate test packs a powerful punch. It's an optimising method that every entrepreneur can and should use to boost conversion rates. Read more to learn how to run your first test. 

Click to Tweet

Multivariate Testing Can Seem Difficult

But it doesn't have to be.

With this information in mind, you'll be well on your way to hitting conversion numbers like never before and finally getting the ROI you deserve. 

If you need any help? Feel free to get in touch with us today to see what we can do.

About the Author

I'm a marketing entrepreneur who loves to work with small business owners and helping them get their message out there.

I cut my teeth in working for a financial startup in the city of London, and have built and run many websites and marketing campaigns for a wide range of clients. I love everything about marketing - from copywriting to developing websites, email newsletters to search engine optimisation.