Archive for Online Testing
Boost your Online Testing Results for Impact: 3 Areas to Review
Posted by: | CommentsIf you are new to online testing and not sure what page or area to test on your website or just need that kick-start to get those testing adrenaline rushes back…
Here are 3 important areas to start pulling data for to get you going (or going again) on the forward path to optimization success.
1. The most visited pages on your website. Things to think about for each page – what’s the pages purpose, what’s the conversion rate, what’s the bounce rate, where are the leaks, what’s the average time spent on the page by your visitors, any coding errors hindering performance, page load time, special plug-ins needed for visitors to get full functionality.
2. Your Conversion points – Pull conversion data for each of your sites conversion points, how much revenue does each conversion point contribute, order each conversion point by revenue from producing the most to the least and look at the opportunities starting at the top of the list – a 100% increase in conversions on a page that only produces $50 won’t produce the same result as a 5% increase on a page that produces $10,000 in revenue – it’s a good place to start.
3. Your most popular visitor paths – Review data for your most popular visitor paths. Where are the leaks that visitors are exiting or straying from your desired end goal that you have designed for them? What are the opportunities to optimize and keep your visitors on the desired path? Can you shorten the path if need be, work on your call-to-actions, add a newsletter signup box, and so on.
4. Bonus – Combinations of the above, i.e the most popular visited page with a conversion point, sorted by lowest conversion percentage with theoretical greatest chance for improvement.
Of course this is not the be all end all of what to look for or what to test in each area, but merely a good refresher for those who need it, or a guiding hand for those confused with all the potential places to start testing first. But remember, it’s important to consider the opportunity costs in testing one area, page, path, etc. versus testing another.
Deconstructing a Marketing Test Hypothesis
Posted by: | CommentsA marketing test hypothesis is a powerful and necessary part of your marketing optimization program when running tests. I am going to take you through creating a simple hypothesis.
A hypothesis clearly states:
- What you are testing
- What your control and experimental groups are
- What outcome you predict will happen (based on an educated judgment)
- What is the alternate outcome
- What you will need to track specifically in order to prove or disprove your test prediction.
Let’s look at a hypothesis more closely.
A hypothesis is clear and specific, testable, and can be proved right or wrong.
Look at these differences between a prediction, a question, and a hypothesis:
A Test Prediction: Not asking for a phone number on my registration form will increase registrations.
A prediction is the outcome you expect – more or less your educated guess of what will happen.
A Test Question: Will not asking for a phone number on my registration form increase registrations?
A Test Hypothesis: Paid Search traffic reaching my registration form that does not ask for a phone number will produce more registrations than Paid Search traffic reaching my registration form that asks for a phone number.
A hypothesis states with conviction what results you expect to see from your test, both from your control and your experimental group – it is here that you will state your test prediction. And since we know that a hypothesis has to be able to be proven either right or wrong, we only have 2 possible outcomes – either my registration form that does not ask for a phone number increases registrations over my control that does ask for it, or it doesn’t produce more registrations(or a tie).
And finally, let’s break down the above marketing test hypothesis to show specifically what it explains:
- Paid Search traffic reaching my registration form that does not ask for a phone number is the experimental group
- Produce more registrations is the outcome we expect from the experiment and what we want to track
- Paid Search traffic reaching my registration form that asks for a phone number is the comparison or control group.
If you wanted to run this test with multiple panels each having a different form field requirements (size), you could in reality replace phone number with less fields:
Paid Search traffic reaching my registration form that has less than 7 fields will produce more registrations than Paid Search traffic reaching my registration form that has 7 fields. “
You are still testing your hypothesis of that less fields will produce more registrations than your control of 7 fields, but you will determine from your testing which length is optimal to rollout with if your hypothesis is true.
With your completed hypothesis you can now execute your marketing test to your website visitors and let them prove or disprove it. If they prove your hypothesis to be true then you did a great job with your hypothesis’s prediction, if they disprove it, you still have learned something (make sure you take away lessons from each test!) – The test is not a failure, just try again after formulating a new hypothesis.
Best Practices & Implementation in Web Site Optimization
Posted by: | CommentsBest practices by definition are ways to do “things” that are more effective or efficient than the other existing ways in order to reach the desired outcome. Every industry has best practices, and almost every task seems to have one as well. Usually they become best practices because a large group has reported that they are following that particular process or procedure and have more effectively or efficiently reached their desired outcome, or even that they have implemented that best practice with no lesser outcome. Many times, best practices are reported and shared by industry associations, seminar speakers, books and reference materials, or even from special reports from your favorite blog or author. From here they spread even more so and become in many instances common place almost as a required implementation to do.
But, this article isn’t really about best practices themselves though, best practices do serve a very import role – even alerting you to what others are currently trying and to foster idea and innovation generation, but rather this is about best practices and marketing optimization – making sure that you don’t blindly implement them and possibly become part of the minority that they didn’t work for and ultimately move further away from your desired goals.
What you’ve heard, read, or been told is a best practice, might not produce the best results for your own website and the traffic that is visiting it – but how do you know?
For example, a certain best practice may be a best practice for 78% of your industry peers, but then this would leave 22% who either didn’t implement it, never heard of it, or it didn’t work for. Not to mention that out of the 78% who did implement it we have no proof of the level of effectiveness it proved to be for each -and are they well versed in reading the results, did they have the right tools to read the results, did they have unknown problems with their tracking, and so on. Now, this isn’t saying that out of that 78% all of them are wrong for using it; maybe 77% were correct in doing so. The point being is that we need to determine how this best practice will work for us and what we are attempting to do and achieve. The proper way to do this is to test the best practice just as you would test anything else.
ColonialCandle.com, according to quotes that appeared on Internet Retailer by Internet marketing manager Katie Fernands, recently ran a multivariate test on a page from their website using Google’s Website Optimizer. This winning page combination was different than their marketing and design departments’ assumptions of not requiring a visitor to scroll down a page too much to view content. In fact their longest page combination increased page conversions by 20% and produced $20,000 in incremental revenue.
Had they not tested (or maybe this was a blind discovery, not sure as I am not aware of the actual test hypothesis they had) the best practice of limiting scrolling on an ecommerce site Colonial Candle would have not discovered that for them a longer page that required scrolling produced better results in producing more conversions resulting in more revenue.
Bottom Line: It’s important to test the best practices for your website rather than just implementing them as gospel in order to make sure that they produce the same results for you that made them known best practices in the first place.
The Other Cost of Cart Abandonment
Posted by: | CommentsHow much is your existing ecommerce cart costing you in cart abandonment? Notice that I didn’t ask you how much revenue you’re potentially losing from those cart abandons.
Take a look at this scenario:
- Each week 1,000 visitors to your online store place an item or items into your ecommerce cart.
- You know that from your research each visitor that adds an item to your shopping cart costs you $12 to acquire.
- You also know from your analytics that your cart abandon rate is 90% (or conversely a 10% conversion rate)
So in this scenario, 900 visitors out of the 1,000 (90%) visitors each week that place an item into your online shopping cart bail and do not purchase, but you have paid for their acquisition anyways. At the $12 per visitor that puts an item into your cart acquisition cost, it’s costing you $10,800 per week ($561,600 per year) for just those visitors that bail out of your cart. How is that so? Continue reading to follow my logic.
So you do some what-if scenarios and believe that with optimization you can get down to an 88% cart abandon rate from your existing 90% abandonment rate (or conversely a 12% conversion rate).
You decide to optimize your shopping cart, and your cart abandonment rate decreases to 87% (better than your prediction of the improved 88% abandonment rate- woohoo!). Now each week your cart is costing you now only $10,400 per week or $542,880 per year – that’s $18,720 less per year, PLUS the additional revenue of those 1,560 paid sales from your cart optimization efforts.
Ok, so you know just as well as I do that that either way you’re still paying the $12 per visitor that puts an item into your cart. So your actual costs haven’t gone down at all, BUT…
Why does this matter and how do you really use this information?
Although your numbers may vary – from the dollar cost of each acquisition that you pay, to the number of visitors that put items into your cart, to your cart abandonment rate. This is purely an exercise in reasoning or a cause for optimization or software upgrading, in other words, an additional metric to prove the value of taking action.
What if your current shopping cart is limiting you to what you can test (or maybe for some reason you can’t perform testing on it) because it’s a third-party application that you have no control over, a legacy cart system that additional programming looks to be costly in time and resources, or some other preventive reason that does not allow you to optimize your cart in the ways that you know you need to in order to increase desired performance.
What if you work for an organization that doesn’t fully believe in the power of testing and optimization? Or, maybe you work for an organization where they are tightening the budget in the current economy and are not interested in investing in optimization.
Your current cart, its current conversion ability and its abandonment rate could be in many ways costing you more in acquisition costs (not including the opportunity costs) than it would cost to fix the problem.
This dollar amount, the cost of cart abandonment, is the cost of leaving your cart as is – the true cost of abandonment of your online shopping cart to you.
How to Document Your A/B or Multivariate Test
Posted by: | CommentsIt’s imperative to document every effort of your online testing and optimization program. Not only to see the progression of improvement over time, but to also have reference for future tests you are planning or questions that may arise from others you are working with.
How I document my A/B and multivariate testing is as follows:
First, I primarily use Excel for all my documentation efforts due its ease of use.
I create a Master Test spreadsheet that serves as a very top-level summary for quick glancing. This spreadsheet is constructed to have column headers for the following:
- Test name – giving each test a descriptive and unique name
- Test date range – documenting the start and finish date of my tests
- Test hypothesis – why I am running this test and why I believe my outcome will be such
- Results – brief description of the result
Each row is for a single test. I then link the Test Name cell for each test to another spreadsheet that is built specifically for that test (If you have ten tests in the Master Test spreadsheet each test would link to its own Individual Test spreadsheet for a total of 10 spreadsheets plus the Master Test Sheet).
Each Individual Test sheet contains multiple tabs and information, and this is where the detail will go throughout the test.
The first tab contains the test information and summary and broken into sections:
- Test name
- Test date range
- Test hypothesis
- What type of test (a/b, multivariate), and how many panels or combinations
- Traffic data (source of traffic, and current traffic stats)
- What type of metrics I will be using to determine the results and how to determine the metrics (is conversion impressions divided by sales, or impressions divided by clicks on a certain button etc?)
- A space to record final metric results (control performed as such, top performers identified individual performed as such)
- Learning’s (both as the test is live and from the results)
- Ideas for future tests based off of this information
- Next actions (will this be rolled out etc.)
- Miscellaneous notes
I also have other tabs in the Individual Test spreadsheet:
- Screenshot of control
- Screenshots of test panels or combinations (depending on how many there are). If there are too many panels or page combinations to take screenshots of, after the test is ended I take screenshots of the top performing test panels for future comparisons)
- Screenshots of Test statistics (When I am using Google Website Optimizer I take daily screenshots of the stats admins. and store in a separate folder, but the final screenshot from the point at which we end the test is stored in the Individual Test spreadsheet – just in case I transcribe something wrong I have an actual reference to go back to.
- Various other tabs as necessary for reference such as more detailed metrics information, etc.
I also keep a folder for each test (with the folder using the test name) that contains my test spec PowerPoint so that I can see all of the elements or options that we are using, analytics data, screenshots of everything-basically anything used from the conception of the test all the way through to the end.
What this enables me to do is at any point in time have a huge history of each test both from a visual standpoint and data-driven standpoint. The Master Test sheet gives me quick access to the individual tests but also a timeline of the testing I have done.
My Testing Documentation Flow
