Archive for October, 2008
Multivariate Testing; Fractional-Factorial, Full-Factorial, Taguchi – Huh? Part 1
Posted by: | CommentsWithin the past year there has been numerous published articles, blog posts, discussions, reports, and I am sure you could even find videos somewhere on the debate between fractional-factorial and full-factorial multivariate testing. Everything from one should only use full-factorial multivariate testing as fractional-factorial is not truly a statistically valid test type, to you should learn to incorporate both into your testing strategies, to everything-else in between.
So what’s the difference between fractional-factorial and full-factorial multivariate testing?
Full-Factorial multivariate testing actually tests all combinations of the options in your test. It takes longer than fractional-factorial since it does go through all of the possible combinations to deliver the results. For instance, if you have 3 page elements with 3 options each, you would have 27 (3×3x3) combinations. If you had 4 page elements, 2 with 3 options, and 2 with 2 options you would have 36 (3×3x2×2) combinations.
Fractional-Factorial multivariate testing does not test all of the combinations of options in your test but instead tests a smaller sampling of them thus allowing the test to run a shorter length to get your results. Most fractional-factorial testing programs then use math to determine these results based on this small sample. A common method used in fractional-factorial testing is the Taguchi Method.
Tim Ash, landing page guru of SiteTuners fame, blogged about the differences in his post titled Taguchi Sucks for Landing Page Testing . He states that:
“The principal drawbacks of fractional factorial methods are:
- Very small test sizes
- Restrictive & inflexible test designs
- Less accurate estimation of individual variable contributions
- Drawing the wrong conclusions
- Inability to consider context and variable interactions”
Fractional-Factorial vs. Full-Factorial: An Ideological War?, an article posted to Omniture’s Industry Insights blog covered their views on these two types of testing styles and why they believe each has their place, specifically when one doesn’t have the necessary amount of time or traffic to run a full-factorial test so that you can get the overall effects compared to the implied effects of each option. You’ll also find a great post by Avinash Kaushik in the comments section.
In the past, I have used fractional-factorial multivariate testing and before rolling out with any test winners have always split tested the best performing page versus our control page. To date, the page of favored options that performed best in our fractional-factorial test has consistently beat our control in the aft split test.
Google’s free Website Optimizer performs full-factorial testing along with the ability to see fractional-factorial data allowing you the ability to choose which data you believe you should follow.
Have opinions or comments on the debate between fractional and full factorial multivariate testing? I would love to hear them!
Optimizing In A Tough Economy – When Paid Search Costs Rise and AOV is Down
Posted by: | CommentsIt’s a no-brainer that it is undeniably more important now so than ever before to be consistently and aggressively optimizing your websites for conversions (both macro and micro-conversions) when the economy is tough as it is right now.
For many online businesses, paid search is one of their primary acquisition sources of new customers, sales and leads. Paid search competition is equally as tough right now as the customer has less money to spend to make their purchases.
Simply and sadly put, it will cost more to acquire the customer who spends less. No one likes to hear those words, ever.
According to this press release from SearchIgnite, who reviewed trends from January 2006 to September 2008, Paid Search growth was strong in the third quarter of 2008 revealing an increase of nearly 27% compared to the same quarter in 2007. The release stated that intra-quarter retail spend showed a decline – and even with a small increase in conversion rates the AOV (average order value) showed a decline.
Two primary solutions with your paid search campaigns are to work on converting more customers that are already coming to your website via this channel or to increase AOV (or both) in your short-term and long-term plans. This is of course involves the incorporation of optimizing your paid search campaigns too.
This is where the familiar “which comes first the chicken or the egg” syndrome comes in. Do you optimize your PPC campaign first to reduce unnecessary spend going out and then optimize your website for better conversions from that traffic? Or, do you optimize your website first in attempt to maximize the conversions from the traffic you are getting and then optimize your PPC campaign from there, or do you do a combination of both at the same time?
Searching the web and depending on whom you read and what theories you subscribe to, you will read many viable reasons why each of these three starting points are the best solution. In the end, most agree the right solution is the one that fits your marketing strategy and goals. But almost all agree that if your testing and optimizing, no matter which end you start with, if results are statistically sounds and financially trustworthy, you are headed in the right direction.
In the end, this is definitely not the time to make guesses on what you think might work to improve your online results (nor was there actually ever a time to make guesses). Rather, this is the time to get proven answers via testing and optimization strategies and to use this data to make the changes needed to increase your online website performance to get closer to your goals.
Using IP Filtering in Google Analytics – The Basics
Posted by: | CommentsIf you are using the free Google Analytics software (or any other analytics package such as Omniture, Webtrends, Coremetrics, etc.) to track where visitors to your website are coming from and how they behave once on your website, make sure that you add a filter to exclude the data gathered from your own IP address. This keeps the data reported to “actual visitors” and disregards your own company’s internal traffic data. You don’t want to take action based on information gathered from tainted data . You or others at your company are probably uploading content and viewing it, reviewing comments, looking at your new graphics, reviewing your test pages, etc., and are adding behavioral and other data to your Google Analytics report that derives from your own time spent on your site.
To make sure that you are excluding your IP address in Google Analytics it’s extremely easy to setup:
- First you need to know your IP address. If you don’t know it, you can get it in less than a second from: WhatIsMyIpAddress.com or WhatsmyIp.org.
- Next, log into your free Google Analytics Account.
There are two ways to proceed,
The Short way:
- In The Website Profiles view, select the large Filter Manager link that appears on the right hand side near the bottom of the page
- Select Add Filter link link on the right hand side
- Name your Filter (i.e Work IP, Home IP, etc.)
- Select your filter type, in this case select Exclude all traffic from an IP address
- Enter in your IP address, but note, you need to do it in the following format:
If your IP address is: 11.222.333.44, you need to enter it in with slashes before each “dot” as follows: 11\.222\.333\.44, the reason being is that the slash keeps the dot from becoming a wildcard and will incorrectly filter other IP addresses (if you are interested in learning more about why you need the slash or other regular expressions you can view the Google Analytics topic page on Regular Expressions).
Google Analytics IP Filter Screen
- Select which websites you want to add the filters to
- Click the finish button and you are done.
The Longer Way (you will need to do the following for each individual Website Profile)
- In The Website Profiles view, select the edit link in the Settings column.
- Scroll down the page to the Filters Applied to Profile section and select the Add Filter link on the right hand side
- Make sure the Add new Filter for Profile radio button is selected
- Name your Filter (i.e Work IP, Home IP, etc.)
- Select your filter type, in this case select Exclude all traffic from an IP address
- Enter in your IP address in the same format as mentioned above.
- Click the finish button and you are done
It’s that easy and that quick and therefore there is no reason why you can’t make this happen right now!
Customer Relationship-Building with Email Newsletters
Posted by: | CommentsI am often surprised at how many local brick-and-mortar businesses don’t offer email newsletters. If they do happen to offer them, they are sent out way too often or much too infrequently and have little to no benefit for the recipient-they don’t build a relationship between you and the reader.
Creating a relationship-building opt-in email newsletter is an absolute necessity and just one of the many must-do ways for you to stay top-of-mind with your target audience (current or potential customers). It’s extremely important not to send out email that has no real value for the recipient. You have to make sure that the content in the email newsletter provides value, and not just stroke your ego. You will over the long-term greatly benefit by providing value to the reader. Make sure the email newsletter keeps your audience interested; the kind of interest that makes them want to tell everyone they know about your newsletter and business which of course spreads word-of-mouth.
As a quick example to explain the concept, imagine you own or manage a local brick and mortar gourmet food shop. You could implement almost immediately a free weekly or biweekly email newsletter to customers containing:
- 1 or 2 recipes where the ingredients are available at your store
- A few special tidbit facts on the origin of the recipe or a new spice
- A cooking tip, maybe in a video format (don’t forget to post it to YouTube, or Howcast)
- A food storage tip
- An interview with a local chef/celebrity/politician about food and or cooking
You could continue to create more value and engagement when you:
- Offer incentives (a gift card to your store) for people to send in a review of recipes they cooked from your newsletter, or a family recipe of their own.
- Feature a spice of the week or of the month and don’t forget to mention to newsletter readers know that they can stop by your store and pick up a sample (no purchase necessary) to try it out, or a special price on it, or a sample of food using it.
- Give readers the opportunity to stop by your store and pick up two additional recipes for free. You could also reverse the offer and give out the two free recipes to those who come to your store and let them know they can get more every month via email by registering for your email newsletter.
The list of possibilities and opportunities is endless. Be creative with your expertise to provide an enewsletter of value, something they want and find worth in receiving and to not delete. Build that relationship with your customers that they won’t get at your competition.
For some this post may be really basic, but way too many miss the point and most importantly the opportunity. You are creating a newsletter to build a relationship that will convert to sales over time, not necessarily today or later this week. You want an email newsletter that is worth it for the reader to open, read, and save in their email box, and if you’re doing it right they are sitting on the edge of their computer seat waiting for the next issue to arrive. It should demonstrate your expertise, draw them back into your store, and even inspire them to forward it to friends.
Keep the expertise up, the sales pitches low-key (unless you’re Kmart) but intertwined – and build that very important value-relationship with your audience.
Increasing Response By Optimizing Your Web Forms
Posted by: | CommentsWhether you actively aware of it or not, your website has an important feature that you just might be overlooking in your marketing optimization regimen – the web form itself. Sure it might be called your shopping cart checkout, your contact form that customers reach you through, your opt-in signup form for your email newsletter, or even your account creation form to access a message board or to make a purchase. All the other parts of your pages may be on your optimization schedule, but if you are creating too much friction or frustration on your web form itself, you could be losing out on conversions.
- Ask yourself, is all the information that I am requesting actually needed right there on this form at this time, or can I request it later. The more information you require now can correlate to a decrease in conversions.
- Is your form button providing a strong call-to-action, or are you still using the old and non-influential “click here” or “submit” button copy. Motivate the customer and give them a reason to finish the required action!
Not sure where to start or would like to get 25 great tips to optimizing the performance of your web form? Check out PalmerWebMarketing’s 25 Web Form Optimization Tips to get yourself headed in the right direction to more webform conversions.
