8 Common Sins of Online Conversion Testing that Organizations Let Happen
ByThree of my conversion optimization colleagues and I had a discussion online the other day that I had proposed about the common “sins” of online conversion testing we see or hear about often in organizations. We came up with about 20 commons “sins” in about 7 minutes that we all agreed upon, and about 40 overall. Below you will find 8 of them in no particular order (with more to come in the future).
8 Common Sins of Online Conversion Testing that Organizations Let Happen:
- When running a multivariate test, after the test ends, not performing a head-to-head testing of the winning page combination and the control. The winning page combination is typically based on a prediction; a head-to-head test will further uncover the true results.
- Having too many people involved in the testing process AFTER the test is given the “go ahead”. Everyone involved should have a purpose otherwise the process slows down.
- Not believing that having no panels perform better than the control is still a win – just of a different kind; but only if you actually extract the knowledge hidden in your “loss”.
- Not setting a concrete conversion goal – know what your test hypothesis is and understand how you will analyze the data ahead of time. Alternate lessons may be and should be learned from a test but it’s vital to know exactly what and why you are testing something in the first place.
- Not allowing a test to run long-enough to accumulate enough conversions.
- Not running the control panel (this happens often) at the same exact time as the test panels.
- Letting personal opinions or biases override data in the results – the reason you test is because you really don’t know what will persuade your actual visitors best.
- No Patience - ending tests too early, or not allowing the process to happen as it should.
As bad as these are, we all agreed we were still happy that organizations have the desire to test!
Have an online conversion testing or optimization sin that you want to share or get off your chest? Let me know in the comments section.
Related posts:
- When is Testing for Conversion Optimization not the Right Thing to Do? There are just certain times when running a multivariate test...
- How to Interpret the Estimated Conversion Rate Range in Google Website Optimizer Interpreting the Estimated Conversion Rate Range Properly in GWO is...
- Landing Page Optimization: Subtraction is the New Addition You’ve already been running numerous tests on your best landing...
- Online A/B Split Test Calculator Are you running an A/B split test for a marketing,...
- Creating a Website that is a True Conversion and Lead Generating Machine Let’s Take a Simple Multiple Choice Test: Question: Who is...
Related posts brought to you by Yet Another Related Posts Plugin.
Leave a Comment
You must be logged in to post a comment.
2 Comments
April 8th, 2009 at 11:02 pm
#3 is a big one for me. Took a while to accept the fact that failures were basically successes, because if it weren’t for testing, I would have just guessed and caused unknown negative impact.
Great post Josh.
April 9th, 2009 at 6:03 am
Great post. As a landing page optimization consultant I completely agree with your 8 sins especially with #6 about not running the control panel at the same time as the test panels. Often businesses that hire me tell me about why they implemented some changes and the tests are often not against the control but the control first, then the test page after and they compare results.
I’d like to add a #9 – Not all tests will have high conversion “winners” everytime.