How and When to Move from A/B to Multivariate Testing

Posted by Richard Patterson - June 13, 2011 - Analytics, SEO, Tools - 10 Comments

Most of us who have a website would love to get it to perform a little better, two obvious ways to do this are to increase traffic to the site or to increase what we get from the existing traffic. By increase what we get from the existing traffic I’m talking about increasing conversions.

The most basic way to increase conversion is to run some A/B testing and is a good starting point for anyone who’s done no form of testing before. However this article is designed to help those who have seen some good results with A/B testing and want to squeeze a little more out of their conversions. If you are new to all forms of conversion testing have a read of this article first which covers the basics.

What is multivariate testing?

Whereas A/B testing tests two pages multivariate testing is able to test two or more pages and changes to multiple elements on those pages in the same test. Multivariate testing allows you to test multiple changes at the same time to find the winning combination of changes that lead to the highest conversion rates. There’s more detailed information here on exactly what multivariate testing is.

Why Multivariate Tests?

Multivariate testing is likely to produce better results but you will usually have to wait longer to see these results (as more variables are tested). Since multivariate testing is able to test multiple elements or factors on a page you are able to find the best combination of changes which result in the highest increase in conversions. Multivariate testing is a natural next step for many website owners who have had some success with A/B testing and want to try and further increase results.

When is A/B testing better?

A/B testing is an easier place to start if you have done no form of testing previously. It also produces faster results which in addition to being great for your bottom line may help give you the confidence to keep testing and move on to more advanced techniques such as multivariate testing at a later date.

A/B testing is also preferable for sites that have low traffic as a smaller number of actions (or conversions), need to be measured to produce statistically significant results compared to multivariate testing.

What to test?

Any call to action or element of a page can be tested, this can include a huge range of things such as:

Headings: Different text, size, color

Images: size, different images, position on page

Buttons: position on page, size, color, writing on the button (e.g. ‘buy now’, ‘proceed’ ‘download’)

Forms: Length, fields, order of fields, content

Website template changes:  color, size of heading text, position of logos, position of accreditation badges (e.g. membership of associations etc).

How to test:

Google’s website optimizer provides a free platform for multivariate testing and has some great tutorials to get you started (you can also A/B test with Website Optimizer).  For larger commercial companies there is a range of companies who specialise in multivariate testing who can advise on what to test for optimal results, help run the testing and analyse the results to maximise performance.

Image credit: Daniel Waisberg

About the Author

Richard Patterson

Interested in all forms of online marketing including link building and SEO, social media, conversion optimization and more. I'm always keen to learn more about online marketing and love to try and figure out what will work well 6 months from now!
9 comments
Custom Papers
Custom Papers

I loved this one! (And I also read this one before book 1, which I loved but not as much as this one)

Deendid
Deendid

I also using A/B. It is one of the basic things they teach you in business-school. What I didn’t know is that Google makes it that easy. I always do it manually. And multi-variate testing is even better. that way you can test even the smallest details. Really interesting to know! Locksmith Toronto prom dresses

template website
template website

Applying statistical techniques to the test process may be the popular foolproof method used by many - done via changing different elements, testing to see if the conversion rate has either increased or decreased.

wood pellet
wood pellet

Great compilation, Kristi! Bookmarking, sphinning and retweeting this one.

flash watch
flash watch

Does not completely solve this problem, in my opinion. We need further discussion.

evergreen homes
evergreen homes

A lot of people find a simple A/B split testing a daunting task. Applying statistical techniques to the test process may be the popular foolproof method used by many - done via changing different elements, testing to see if the conversion rate has either increased or decreased.

By Files
By Files

I’ve used A/B testing many times. It is one of the basic things they teach you in business-school. What I didn’t know is that Google makes it that easy. I always do it manually. And multi-variate testing is even better. that way you can test even the smallest details. Really interesting to know!

cold room
cold room

Your rank in the search results does not always equate to sales and conversions, in my opinion.

Eiendom i Tyrkia
Eiendom i Tyrkia

I agree with your Blog and I will be back to check it more in the future so please keep up your work. I love your content & the way that you write. It looks like you’ve been doing this for a while now, how long have you been blogging for? My recent post Beautiful fully furnished apartment in alanya, Alanya / Turkey