How and When to Move from A/B to Multivariate Testing

Posted by Richard Patterson - June 13, 2011 - Analytics, SEO, Tools - 10 Comments

Most of us who have a website would love to get it to perform a little better, two obvious ways to do this are to increase traffic to the site or to increase what we get from the existing traffic. By increase what we get from the existing traffic I’m talking about increasing conversions.

The most basic way to increase conversion is to run some A/B testing and is a good starting point for anyone who’s done no form of testing before. However this article is designed to help those who have seen some good results with A/B testing and want to squeeze a little more out of their conversions. If you are new to all forms of conversion testing have a read of this article first which covers the basics.

What is multivariate testing?

Whereas A/B testing tests two pages multivariate testing is able to test two or more pages and changes to multiple elements on those pages in the same test. Multivariate testing allows you to test multiple changes at the same time to find the winning combination of changes that lead to the highest conversion rates. There’s more detailed information here on exactly what multivariate testing is.

Why Multivariate Tests?

Multivariate testing is likely to produce better results but you will usually have to wait longer to see these results (as more variables are tested). Since multivariate testing is able to test multiple elements or factors on a page you are able to find the best combination of changes which result in the highest increase in conversions. Multivariate testing is a natural next step for many website owners who have had some success with A/B testing and want to try and further increase results.

When is A/B testing better?

A/B testing is an easier place to start if you have done no form of testing previously. It also produces faster results which in addition to being great for your bottom line may help give you the confidence to keep testing and move on to more advanced techniques such as multivariate testing at a later date.

A/B testing is also preferable for sites that have low traffic as a smaller number of actions (or conversions), need to be measured to produce statistically significant results compared to multivariate testing.

What to test?

Any call to action or element of a page can be tested, this can include a huge range of things such as:

Headings: Different text, size, color

Images: size, different images, position on page

Buttons: position on page, size, color, writing on the button (e.g. ‘buy now’, ‘proceed’ ‘download’)

Forms: Length, fields, order of fields, content

Website template changes:  color, size of heading text, position of logos, position of accreditation badges (e.g. membership of associations etc).

How to test:

Google’s website optimizer provides a free platform for multivariate testing and has some great tutorials to get you started (you can also A/B test with Website Optimizer).  For larger commercial companies there is a range of companies who specialise in multivariate testing who can advise on what to test for optimal results, help run the testing and analyse the results to maximise performance.

Image credit: Daniel Waisberg

About the Author

Richard Patterson

Interested in all forms of online marketing including link building and SEO, social media, conversion optimization and more. I'm always keen to learn more about online marketing and love to try and figure out what will work well 6 months from now!