Welcome to Josh Baker's Practical Advice for Optimizing Your Internet Marketing blog. Here you will find internet marketing optimization and online strategy articles full of tips, tricks, discussions, and thoughts to help you take your marketing and business to the next level of success.

AB Testing or Multivariate Testing – The Laymans Explanation


In most instances of classic A/B testing, you alter only one page element at a time in order to determine if that particular change is responsible for the increase, decrease, or in some cases neither in the results achieved. Which panel achieved more click-throughs, Panel (page) A with the red button or panel B with the Blue button? The panel which featured Headline A or the panel that featured Headline B? The black and white image or the color image increased response? The registration form on the left-hand side or on the right-hand side of the page increased submissions? You could of course test two completely different pages against each other, but it would be impossible to determine which elements were responsible for the increase or the decrease in response. You could only determine then if either panel A or panel B was the winner or loser.

Classic Multivariate Testing allows you to simultaneously test multiple changes to a webpage in order to uncover the best possible combination of page elements that reach your goal for that page. You can now not only test the red button versus the blue button, but also determine if the blue button with Headline A out performs all other combinations of these 2 elements. Now this may seem like a simple and common test because we are only testing 2 variables, with 2 options each. But now imagine testing at the same time:

  • 3 different registration form Submit button colors
  • 3 different registration forms headlines
  • 3 different page headlines
  • 3 different main copy lengths
  • 3 hero images

With classic A/B testing, this would take many tests and amounts of time to achieve the results. With Multivariate Testing you can uncover the best page combination of all of these different variables and options in one single multivariate test (considering you have the traffic volume to do so at once).

Although the preparation and setup of a multivariate test can take in many cases more time and conceptualization than the much simpler and straightforward A/B test, you are able to achieve results quicker than running A/B test after A/B test to get to the same answer to your testing question.

Which style of testing is the right one for you? It all depends on what your goals are, what tools you have (Google Website Optimizer is free), and what type of traffic you have available.

Categories : Online Testing

Leave a Comment