StayOnSearch

Dos and Don’ts of A/B testing

Kevin Harris May 25, 2011

Even if your company’s website is perfectly fine and you are getting traffic and have a high page ranking, you may want to test the waters and see if you can improve your website’s statistics. The best way to do this is to do A/B testing. A/B testing lets you know if a different version of your website is better at drawing visitors than your current one. With classic A/B testing a company would send out two different versions of pamphlets to their direct mailing list. Half would get the old version and half would get a new version. This way the company could see which one brought in the most sales and generated the most interest. A/B testing these days holds true to the same principle, just with websites. This can be a super marketing tool, if done right. And, like with most things, there is a right and a wrong way to do it.

Don’t test too many variations at once

If you test too many different version of your company website, it will be very hard to narrow down which one did the best. The more versions of your website that you test, the larger you need your sample audience to be. And if you have way too many websites to test, you may not be able to get a sample size large enough to do your testing. It’s best to stick to two or three sites at a time so that you have more control and an easier time of testing.

Do know what you are testing for

Are you interested in increasing your sale or just getting more people to your website? Or maybe you want to know which version will entice the most page clicks per visit. Know what you are testing for before you start will make the process and the data analysis a lot easier.

Don’t change too many elements of the website

If you change the header, font, background color, add graphics and the new website gets some more hits it will be very hard to know exactly what element attracted the additional attention. This can become a huge issue if you are testing more than one new site and you are reusing some of the elements. If you are looking at completely overhauling your website this may not be totally bad. But if you just wanted to make one or two improvements this approach will make it next to impossible to know which ones to change.

Test all the sites at the same time

Simultaneous testing is the key to A/B testing. If you direct everyone to the original site on Tuesday morning and then direct everyone to the new site on Wednesday night, even if the new site gets twice as many hits, the data will be inconclusive and unusable. Why? If more people are apt to surf the web on a Wednesday night rather than a Tuesday morning, it would only make sense that the new site would get more hits. For a true test you need to randomly direct visitors to one of the sites you are testing at teh same time.

Have a large enough sample size

If your sample size is too small, any data collected will be insignificant and unusable. Ideally you want to have as many people visiting your site as possible. The larger the sample size, the better your results and the more concrete the data collected will be. If you find that you aren’t getting a large enough sample of visitors, you will need to either run the A/B test for a longer period of time or test fewer sites.

Tagged with:
2 comments
morrison real estate
morrison real estate

A/B testing is a common strategy for many successful websites. This strategy has been proven to be very effective indeed. Every web owner, especially those who are new to the business, should consider this.

Ivan
Ivan

Another suggestion: change the person doing the testing. You’d be very surprised in the way people interpret results and skewer them to suit their own needs.

Login to your account

Can't remember your Password ?

Register for this site!