Archive for Marketing Basics
Deconstructing a Marketing Test Hypothesis
Posted by: | CommentsA marketing test hypothesis is a powerful and necessary part of your marketing optimization program when running tests. I am going to take you through creating a simple hypothesis.
A hypothesis clearly states:
- What you are testing
- What your control and experimental groups are
- What outcome you predict will happen (based on an educated judgment)
- What is the alternate outcome
- What you will need to track specifically in order to prove or disprove your test prediction.
Let’s look at a hypothesis more closely.
A hypothesis is clear and specific, testable, and can be proved right or wrong.
Look at these differences between a prediction, a question, and a hypothesis:
A Test Prediction: Not asking for a phone number on my registration form will increase registrations.
A prediction is the outcome you expect – more or less your educated guess of what will happen.
A Test Question: Will not asking for a phone number on my registration form increase registrations?
A Test Hypothesis: Paid Search traffic reaching my registration form that does not ask for a phone number will produce more registrations than Paid Search traffic reaching my registration form that asks for a phone number.
A hypothesis states with conviction what results you expect to see from your test, both from your control and your experimental group – it is here that you will state your test prediction. And since we know that a hypothesis has to be able to be proven either right or wrong, we only have 2 possible outcomes – either my registration form that does not ask for a phone number increases registrations over my control that does ask for it, or it doesn’t produce more registrations(or a tie).
And finally, let’s break down the above marketing test hypothesis to show specifically what it explains:
- Paid Search traffic reaching my registration form that does not ask for a phone number is the experimental group
- Produce more registrations is the outcome we expect from the experiment and what we want to track
- Paid Search traffic reaching my registration form that asks for a phone number is the comparison or control group.
If you wanted to run this test with multiple panels each having a different form field requirements (size), you could in reality replace phone number with less fields:
Paid Search traffic reaching my registration form that has less than 7 fields will produce more registrations than Paid Search traffic reaching my registration form that has 7 fields. “
You are still testing your hypothesis of that less fields will produce more registrations than your control of 7 fields, but you will determine from your testing which length is optimal to rollout with if your hypothesis is true.
With your completed hypothesis you can now execute your marketing test to your website visitors and let them prove or disprove it. If they prove your hypothesis to be true then you did a great job with your hypothesis’s prediction, if they disprove it, you still have learned something (make sure you take away lessons from each test!) – The test is not a failure, just try again after formulating a new hypothesis.
Best Practices & Implementation in Web Site Optimization
Posted by: | CommentsBest practices by definition are ways to do “things” that are more effective or efficient than the other existing ways in order to reach the desired outcome. Every industry has best practices, and almost every task seems to have one as well. Usually they become best practices because a large group has reported that they are following that particular process or procedure and have more effectively or efficiently reached their desired outcome, or even that they have implemented that best practice with no lesser outcome. Many times, best practices are reported and shared by industry associations, seminar speakers, books and reference materials, or even from special reports from your favorite blog or author. From here they spread even more so and become in many instances common place almost as a required implementation to do.
But, this article isn’t really about best practices themselves though, best practices do serve a very import role – even alerting you to what others are currently trying and to foster idea and innovation generation, but rather this is about best practices and marketing optimization – making sure that you don’t blindly implement them and possibly become part of the minority that they didn’t work for and ultimately move further away from your desired goals.
What you’ve heard, read, or been told is a best practice, might not produce the best results for your own website and the traffic that is visiting it – but how do you know?
For example, a certain best practice may be a best practice for 78% of your industry peers, but then this would leave 22% who either didn’t implement it, never heard of it, or it didn’t work for. Not to mention that out of the 78% who did implement it we have no proof of the level of effectiveness it proved to be for each -and are they well versed in reading the results, did they have the right tools to read the results, did they have unknown problems with their tracking, and so on. Now, this isn’t saying that out of that 78% all of them are wrong for using it; maybe 77% were correct in doing so. The point being is that we need to determine how this best practice will work for us and what we are attempting to do and achieve. The proper way to do this is to test the best practice just as you would test anything else.
ColonialCandle.com, according to quotes that appeared on Internet Retailer by Internet marketing manager Katie Fernands, recently ran a multivariate test on a page from their website using Google’s Website Optimizer. This winning page combination was different than their marketing and design departments’ assumptions of not requiring a visitor to scroll down a page too much to view content. In fact their longest page combination increased page conversions by 20% and produced $20,000 in incremental revenue.
Had they not tested (or maybe this was a blind discovery, not sure as I am not aware of the actual test hypothesis they had) the best practice of limiting scrolling on an ecommerce site Colonial Candle would have not discovered that for them a longer page that required scrolling produced better results in producing more conversions resulting in more revenue.
Bottom Line: It’s important to test the best practices for your website rather than just implementing them as gospel in order to make sure that they produce the same results for you that made them known best practices in the first place.
The Value of Upper Funnel Keywords in PPC
Posted by: | CommentsUpper Funnel Keywords in paid search (or even in organic search by definition) are those keywords that do not bring in visitors that convert in a traditional sense into an immediate purchase or lead, but rather are the keywords that convert the potential customer to the next stage of the “interest cycle”. These Upper Funnel Keywords typically bring potential customers or leads into your website that have not made up their mind to make a purchase (or fill out a form or take a trial) because they are still in their “considering their options stage” of decision making and are possibly not yet familiar with you or in some instances not familiar enough with you to complete a conversion during that visit.
Example:
You are going on vacation to a cold climate (let’s say the North Pole). You know you need to buy something to keep yourself warm, but you’re not sure exactly what options are out there because you live in a year round warm climate. So you do a search on the keyword phrase extreme cold weather coats. You end up at a website by clicking on a paid search ad that appeared when you typed in the phrase and notice that they have a section for cold climate coats and even list the temperatures they can withstand. You think to yourself, wow this is great, now I have a better idea of what’s available and I kind of like Brand X Model 4567, but I am not going on the vacation for another 4 months so I am not going to purchase anything today because I don’t have the extra money. Forty-five days later you win $200 on a scratch-off ticket that you found and decide that you are ready to purchase a coat with your extra money, you do a search on Brand X Model 4567 since you knew the exact coat you wanted to buy now and end up back on the same website as before and purchase the coat.
The difficulty in measuring the value of these Upper Funnel Keywords is that they don’t produce single visit conversions -you can’t see the whole picture of entrance to conversion in your analytics data in a linear fashion.
For instance, a single visit conversion would show start to finish in one visit from entering your site to making the purchase. Here you easily have the whole picture from the PPC keyword that triggered the ad that they clicked on to enter your website and their entire path to the purchase.
With Upper Funnel Keywords, a typical scenario would be that the visitor arrives at your site from these keywords or phrases, looks around, leaves, comes back another time reads more information, exits your site again, then finally comes back a 3rd time and makes a purchase.
Most only know how to measure the keywords that produce single visit conversions and thus deem these Upper Funnel Keywords more or less valueless because their value isn’t easily seen in a typically known fashion. The ROI isn’t easily visible.
In the example given earlier, the keyword phrase you searched on first, extreme cold weather coats, was the Upper Funnel Keyword phrase. You didn’t purchase during that visit of your initial search, but you did eventually go back and purchase based on the information you learned during that first visit. Had another website come up with a different but possibly similar featured coat that you liked, you would have purchased from there and a different coat. So there was definite value in that first search as it gave you the information you needed to make a decision, but you just didn’t purchase then. Now imagine if you were only looking at single session conversions, you would only be able to confirm that Brand X Model 4567 was a valuable keyword because a sale was associated with it. But in reality, without having a presence in paid search for extreme cold weather coats a sale wouldn’t have been made – thus showing the importance of being able to look at multi-session conversions to contribute back value (and ROI) to the Upper Funnel Keywords.
Avinash Kaushik on his Occam’s Razor blog this week made a powerful and very instructionally clear post on how to measure the success of Upper Funnel Keywords and I suggest you read his post for the details on how to do so.
The Other Cost of Cart Abandonment
Posted by: | CommentsHow much is your existing ecommerce cart costing you in cart abandonment? Notice that I didn’t ask you how much revenue you’re potentially losing from those cart abandons.
Take a look at this scenario:
- Each week 1,000 visitors to your online store place an item or items into your ecommerce cart.
- You know that from your research each visitor that adds an item to your shopping cart costs you $12 to acquire.
- You also know from your analytics that your cart abandon rate is 90% (or conversely a 10% conversion rate)
So in this scenario, 900 visitors out of the 1,000 (90%) visitors each week that place an item into your online shopping cart bail and do not purchase, but you have paid for their acquisition anyways. At the $12 per visitor that puts an item into your cart acquisition cost, it’s costing you $10,800 per week ($561,600 per year) for just those visitors that bail out of your cart. How is that so? Continue reading to follow my logic.
So you do some what-if scenarios and believe that with optimization you can get down to an 88% cart abandon rate from your existing 90% abandonment rate (or conversely a 12% conversion rate).
You decide to optimize your shopping cart, and your cart abandonment rate decreases to 87% (better than your prediction of the improved 88% abandonment rate- woohoo!). Now each week your cart is costing you now only $10,400 per week or $542,880 per year – that’s $18,720 less per year, PLUS the additional revenue of those 1,560 paid sales from your cart optimization efforts.
Ok, so you know just as well as I do that that either way you’re still paying the $12 per visitor that puts an item into your cart. So your actual costs haven’t gone down at all, BUT…
Why does this matter and how do you really use this information?
Although your numbers may vary – from the dollar cost of each acquisition that you pay, to the number of visitors that put items into your cart, to your cart abandonment rate. This is purely an exercise in reasoning or a cause for optimization or software upgrading, in other words, an additional metric to prove the value of taking action.
What if your current shopping cart is limiting you to what you can test (or maybe for some reason you can’t perform testing on it) because it’s a third-party application that you have no control over, a legacy cart system that additional programming looks to be costly in time and resources, or some other preventive reason that does not allow you to optimize your cart in the ways that you know you need to in order to increase desired performance.
What if you work for an organization that doesn’t fully believe in the power of testing and optimization? Or, maybe you work for an organization where they are tightening the budget in the current economy and are not interested in investing in optimization.
Your current cart, its current conversion ability and its abandonment rate could be in many ways costing you more in acquisition costs (not including the opportunity costs) than it would cost to fix the problem.
This dollar amount, the cost of cart abandonment, is the cost of leaving your cart as is – the true cost of abandonment of your online shopping cart to you.
Standard Deviation and Marketing – How, Why?
Posted by: | CommentsWhat Is Standard Deviation?
Standard deviation is used to measure the average difference between the individual numbers that make up your data sets’ arithmetic mean (mean being the average or center) and their mean value.
Example 1:
You have 4 numbers; 7,10,6, and 7 again, their mean ((7+10+6+7 =30)/4) = is 7.5, but their standard deviation is 1.5 (I’ll explain later how to calculate the standard deviation). So the average of the numbers is 7.5 plus or minus 1.5. You will usually see it reported as 7.5 +/- 1.5. The addition of the standard deviation number enables you to tell a more complete story of the data spread in the data set allowing viewers to see the average difference between the individual numbers that make up your mean and their mean value.
Example 2:
Below is a more of an extreme example, but it shows you the importance of knowing what the standard deviation is when reporting on an average of a data set:
Now let’s assume you are doing a poll of all 4 people that work in your small office (including yourself) because you want to know the average number of pens everyone in your office has at their desk. The answers you get are 5,7,10, and 200 pens. Their mean (or average) is 55.5, but their standard deviation is 83.4.
By reporting it as the workers in your office have an average of 55.5 plus or minus 83.4 pens really lets the reader of the data know that data set has a large spread. Although the average is 55.5 pens, it really doesn’t give any insight to the fact that 1 of your coworkers has 200 pens while the rest have less than 10 pens.
What happens if all the numbers in your data set are the same? Well, if all 4 people reported that they had 2 pens each, then your average or mean would be 2 pens with a standard deviation of 0 because all of the numbers that make up your data set are the same as the average number itself, so there is no deviation from the average for the number of pens each individual has.
Why you need to understand standard deviation in Marketing and Optimization?
So now you’re thinking, OK, so how is this applicable to Marketing and Optimization?
Let’s say you want to know the Average Order Value (AOV) of a particular channel that sends visitors to your eCommerce site (some people dont like to use standard deviation when working with value, so you could replace AOV with average order size in this example). For simplicities sake (to keep this explanation short- so go along with this) let’s pretend that you have 10 orders (yes this is small, but again to keep this simple) during the period you are viewing – 4 orders at $99, 1 order at $1,499, and 5 orders at $5. Your AOV is $192 (all 10 numbers added up and divided by 10), but the standard deviation is $437.91.
Telling someone that you’re AOV is $192 really does disservice to the real truth of your order values. By stating that your AOV is $192+/- $437 lets the viewer take a step back and realize that there is an order or orders that are spread out far from the mean. Yes, some people will remove the outliers, etc., etc., but if you do that, you should provide that as an additional metric alongside the mean and the accompanying standard deviation – but that conversation is better set for a different time (kind of like discussing politics as you will get all sides and opinions).
Can you think of other areas where knowing this in your marketing would really help understand the numbers better? Rarely do you have such small data sets that you want to base decisions on for marketing’s sake as I have presented here for simplicities sake. But imagine if you had 1,000 numbers to report on that were all over the place with varying spreads? How about comparing performance of various test panel results in your online testing?
The closer the individual numbers are from your mean, the smaller the standard deviation will be. And, the further the individual numbers are from your mean, the larger the standard deviation will be.
How to Calculate Standard Deviation in Excel
In Excel to get to standard deviation is for the most part really simple:
If your data is from a SAMPLE, use this Excel standard deviation formula (a sample is a portion of the total population, i.e. 300 employees chosen randomly, 500 orders chosen randomly):
=STDEV(Cell Range)
I should note that you don’t want to actually type in Cell Range, but put in the cell range or individual cells that you are your data set: so if you have your data in cells A1, A2, A3, and A4, then you would use the formula =STDEV(A1:A4) to get your standard deviation.
If your data is from a POPULATION, use this Excel standard deviation formula (a population is the total population, i.e. all employees at work, all orders received):
=STDEVP(Cell Range)
I should note again that you don’t want to actually type in Cell Range, but put in the cell range or individual cells that you are your data set: so if you have your data in cells A1, A2, A3, and A4, then you would use the formula =STDEVP(A1:A4) to get your standard deviation.
Hand Calculating Standard Deviation (not advised for large data sets)
- 1. Add up all the numbers in your data set to get a total
- 2. Take that total and divide it by how many numbers you have in your data set (this is your mean)
- 3. Subtract the difference of each number from the mean and square each difference
- 4. Add each of the squared differences together
- 5. Take the sum of the squared differences in step 4 and divide it by how many numbers you have in your data set (this number is your variance)
- 6. Take the square root of the variance
- 7. The answer is your standard deviation
Online Standard Deviation Calculator
A nicely done online standard deviation calculator can be found at Math is Fun Standard Deviation Calculator page. Definitely take some time to play with this; well I think its fun at least.
So the next time someone gives you some “average” ask them what the standard deviation is and you probably will throw them what they think is a curve ball, but it will give you much clearer insight into what the real info is behind that average. And for your own marketing purposes, it will give you a better understanding of your data and those who need to view it (after of course explaining what standard deviation is and why it’s important!).
