Sample Scenario:

• You want to add a page with a video or demo to your web site.
• You believe that it is important for visitors to view as much as possible of the video.
• You have 2 videos to test against each other, and your test goal is to determine which one of your videos keeps more visitors watching (engaged) for at least a certain period of time and plan to  keep the one that engages viewers more than the other.

In this scenario, since your test goal is to determine which video users are more engaged in watching for at least a certain period of time you will therefore want a conversion to trigger and to be recorded in GWO after that set period of predetermined minimum viewing time.

Back to the Scenario:

• You want a conversion to trigger if a visitor in the experiment views at least 20 seconds of your 1 minute videos.

This is easily done by using Time On Page as your conversion goal. What basically happens is that if the visitor stays on the page or in this scenario views the video for that certain period of time that you have predetermined; the setTimeout JavaScript function (in the script given further down in the article) executes the script that reports a conversion in Google Website Optimizer.

To do this, use this time on page goal tracking script and place it at the bottom of the page that has your control script and test page tracking script. Change the red bold faced font in the script below to the amount of time you want the visitor to have reached before a conversion is triggered. The number is represented in milliseconds where 1000 milliseconds equal 1 second – so in this scenario since we want 20 seconds of viewing time to trigger a conversion, simple multiplication of 1000 milliseconds x 20 = 20000. Therefore if the viewer views the video for AT LEAST 20 seconds a conversion is recorded. You will replace the xxxx’s in blue below with the data given to you in the control script and test page tracking script (it references your GWO account and this test).

<script>
if(typeof(_gat)!=’object’)document.write(’<sc’+'ript src=”http’+ (document.location.protocol==’https:’?’s://ssl’:'://www’)+ ‘.google-analytics.com/ga.js”></sc’+'ript>’)
</script>

<script>
setTimeout(’_gat._getTracker(”UA-xxxxx-x“)._trackPageView(”/xxxxxxxxxx/goal”)’, 20000);
</script>

Note: Since the conversion is triggered in this instance by a viewer who stays on your page for a certain minimum period of time, and if your page has more than just the video (such as news, photos, or articles) and a visitor stays for the minimum period of time doing other “things”, the conversion will still trigger. You either need to set your test to be just for the video (requiring a more advanced setup), or have nothing else on the page but the video – and by nothing else I am referring to other pieces of content that could potentially keep people on that page doing something other than viewing the video.

Related posts:

1. How to Interpret the Estimated Conversion Rate Range in Google Website Optimizer Interpreting the Estimated Conversion Rate Range Properly in GWO is...

It’s easy to get initially excited when you see that one of your test panels in Google Website Optimizer has a higher Estimated Conversion Rate than that of your control panel as presented on the Combination Report page. This may even lead to you believe you can end the test (prematurely I may add).

Unfortunately, just looking at the Conversion Rate number given to you by Google in the bold type font isn’t enough, you most certainly need to do a little bit of visualization to really have a better understanding of what is going on and how they are performing against each other.

Whether you are running an A/B test or a multivariate test, this is important to know for either – the number they give you is a conversion RATE RANGE. Many people mistakenly look at just the number given and do not visualize the full conversion range given along with it (done so with simple addition and subtraction of the number given next to the estimated conversion rate after  the plus and minus sign). This range is based on the observed conversion rate of during the experiment thus far. Not factoring this in can lead to many people ending or wanting to end tests before they are truly ready to be ended. For example,

Estimated Conversion Rate

• Test Panel – 6.0% +/- 1.0%
• Control Panel – 5.5% +/- 1.0%

Reading and interpreting this correctly would actually tell you that the:

• Test Panel is converting in the range of 5.0% to 7.0%, and the
• Control Panel is converting in the range of 4.5% to 6.5%

This being true, their conversion rate ranges are overlapping each other.  Visualizing this information shows you the overlap much more clearly as shown below:

So, is the test panel really converting better than the control? Potentially the control panel could still be converting better – it’s not clear enough to make a decisive or correct decision at this point.

Now think of a scenario such as two panels having a higher conversion rate and much larger conversion rate range:

• Test – 55.0% +/- 8.0% (conversion rate range of 47.0% to 63.0%)
• Control – 52.0% +/- 7.0% (conversion rate range of 45.0% to 59.0%)

Some may feel that the test panel is the winner and so it’s ok to stop the test and roll out with the test panel…BUT…

As the test goes on for a longer period of time, you should find the conversion rate range getting “tighter” and the above test potentially ending in such as a way as:

• Control panel converting at 58% +/- 1 .0%, and the
• Test panel converting at 47.0% +/- 1.0%.

In this ending scenario the control panel is actually the winner not the test panel as we saw mistakenly earlier on. Ending the test too early certainly changes the story drastically as does the results could differ as well!

I know sometimes it’s exciting to see your test panel winning, but patience is certainly of high importance in A/B and multivariate testing in order to make sure your tests and results are always moving forward correctly.

Related posts:

1. Testing Video Engagement Using Google Website Optimizer Using Minimum Viewing Time as a A/B Test Conversion Goal...

8 Common Sins of Online Conversion Testing that Organizations Let Happen:

1. When running a multivariate test, after the test ends, not performing a head-to-head testing of the winning page combination and the control. The winning page combination is typically based on a prediction; a head-to-head test will further uncover the true results.

 

2. Having too many people involved in the testing process AFTER the test is given the “go ahead”. Everyone involved should have a purpose otherwise the process slows down.

 

3. Not believing that having no panels perform better than the control is still a win – just of a different kind; but only if you actually extract the knowledge hidden in your “loss”.

 

4. Not setting a concrete conversion goal – know what your test hypothesis is and understand how you will analyze the data ahead of time. Alternate lessons may be and should be learned from a test but it’s vital to know exactly what and why you are testing something in the first place.

 

5. Not allowing a test to run long-enough to accumulate enough conversions.

 

6. Not running the control panel (this happens often) at the same exact time as the test panels.

 

7. Letting personal opinions or biases override data in the results – the reason you test is because you really don’t know what will persuade your actual visitors best.

 

8. No Patience - ending tests too early, or not allowing the process to happen as it should.

As bad as these are, we all agreed we were still happy that organizations have the desire to test!

Have an online conversion testing or optimization sin that you want to share or get off your chest? Let me know in the comments section.

Related posts:

1. How to Interpret the Estimated Conversion Rate Range in Google Website Optimizer Interpreting the Estimated Conversion Rate Range Properly in GWO is...

Why is this so? Although each page, situation, and context is many times unique, a few of the more common reasons for the improvement in conversions include:

1) Removing distractions that enable the visitor to more clearly focus on your desired page goal.

2) Reducing the friction that forces the visitor to contemplate if the desired action is worth what is being asked of them to give in return.

3) Replacing confusing elements that prevent the visitor from understanding if they are on the correct page or even knowing what they are supposed to do next.

A few broader ranged ideas to consider include:

• Removing to clear up page real estate
• Removal to speed up page load time
• Removal of potential road blocks or barriers

More detailed removal considerations include:

• Removal of parts/all of navigation
• Removal of sections of copy
• Removal of unnecessary graphics
• Removal of just the large file size images
• Removal of flash elements (or those that require plug-ins or longer load time)
• Removal of non-vital third party java-scripts
• Removal of non-essential registration form fields
• Removal of traffic-leaks
• Removal of premiums or special offers

These should be enough start ideas to get you thinking in the right direction when you are looking at your landing page. You undoubtedly will develop various unique hypotheses for doing these (or any other “removal” ideas) based upon your own site’s data you have extracted and analyzed-or even from basic usability knowledge. The end goal is ultimately almost always the same – to uncover what page elements are negatively impacting your page’s ability to do its job properly so that you can fix them to increase the level of success your site achieves. Remember, removal testing doesn’t have to be done in isolation; removal can always be a part of any test when it’s appropriate to do so as judged by you.

No related posts.

Here are 3 important areas to start pulling data for to get you going (or going again) on the forward path to optimization success.

1. The most visited pages on your website. Things to think about for each page – what’s the pages purpose, what’s the conversion rate, what’s the bounce rate, where are the leaks, what’s the average time spent on the page by your visitors, any coding errors hindering performance, page load time, special plug-ins needed for visitors to get full functionality.

2. Your Conversion points – Pull conversion data for each of your sites conversion points, how much revenue does each conversion point contribute, order each conversion point by revenue from producing the most to the least and look at the opportunities starting at the top of the list – a 100% increase in conversions on a page that only produces $50 won’t produce the same result as a 5% increase on a page that produces $10,000 in revenue – it’s a good place to start.

3. Your most popular visitor paths – Review data for your most popular visitor paths. Where are the leaks that visitors are exiting or straying from your desired end goal that you have designed for them?  What are the opportunities to optimize and keep your visitors on the desired path? Can you shorten the path if need be, work on your call-to-actions, add a newsletter signup box, and so on.

4. Bonus – Combinations of the above, i.e the most popular visited page with a conversion point, sorted by lowest conversion percentage with theoretical greatest chance for improvement.

Of course this is not the be all end all of what to look for or what to test in each area, but merely a good  refresher for those who need it, or a guiding hand for those confused with all the potential places to start testing first. But remember, it’s important to consider the opportunity costs in testing one area, page, path, etc. versus testing another.

Related posts:

1. How to Interpret the Estimated Conversion Rate Range in Google Website Optimizer Interpreting the Estimated Conversion Rate Range Properly in GWO is...