Author Archive
Interpreting Google Website Optimizer’s Page Sections Report
Posted by: | CommentsGoogle Website Optimizer, the free multivariate and AB testing tool makes available to you two results reports when you are running a multivariate test; the Combinations report, and the Page Sections report. The Combinations report is easy to read and interpret for the most part (for simplicity purposes here, the pages are listed in order of best performance, and a statistically significant winner with a 95+% confidence level over the control will be highlighted in green), but the Page Sections report many people incorrectly use the data that is presented or are confused on what the report is actually providing. You MUST know how to correctly interpret this report; otherwise you could incorrectly make decisions from the data that can cost you potentially big-time down the road.
The Combinations report provides you with the performance data for all of your page combinations that you are testing and compares each page combination to your original page (the control) in order for you to determine what test page combinations are outperforming the original.

The Page Sections report on the other hand displays all of your page sections that you are testing and the elements within each section in order to show you the best performing page element for each particular section.

Here is the scoop on the Page Sections report.
Many times, when you are running a test you will notice that the best performing elements according to the Page Sections report are not the same elements that are present in your winning page combination. It’s not as simple as looking at the different sections and choosing all the “best” elements according to Google Website Optimizer Page Sections report and then rolling out a page that contains these elements. The reason being is that the performance of the elements in each section being tested as shown in the Page Sections report are not being calculated and presented in context with the other sections. By this I mean that the results of each element in each section are presented by how they performed against the other elements in that same page section. It’s not taking into account the context of all the other elements on the page when showing you the best performers. To see which the best elements are in proper context, view your Combinations report and look at the individual elements in your top performing page combination.
Confused?
Let me give you an example. If you are testing a main headline, and have 3 different versions:
- Control/original headline
- Test headline # 1
- Test headline #2
And the report shows that test headline # 1 is the best performing element in that area, it’s only reporting to you that between those 3 headlines, headline #1 is the best performer, but it is not taking into consideration how it interacts with the other elements on the page. It’s not saying that for the page you are testing, headline #1 is the best to use for the roll-out page, but instead – between those 3 headlines, headline #1 is the best performing if you are comparing just the performance of the headline itself. Headline #1 might perform the best out of the 3 headlines when compared to each other out of context with the rest of the page elements, but Headline #2 might perform the best in combination with all the other elements and therefore is the one you want to roll out with (but not necessarily reported this way in the Page Sections report). So please, don’t use the Page Sections report to pick out which elements to roll-out live with.
The Relevance Rating graphic (currently found to the left of each page section), alerts you to how much impact each section had on your test (as defined by your test’s conversion goal). It’s presented as a range, from 0 to 5, with 0 representing that the section had virtually no impact on conversions and a 5 as having a high impact on page conversions.
So while the Page Sections report is important to look at to understand what is happening between the different elements in the page sections themselves, and the Relevance Rating is important to look at to understand what page sections are providing what level of impact to page conversions, you should ideally only use this information to help you gain more insights into the test. Insights that will also give you further help in planning future testing of the page.
However, you will only want to roll-out the panel that is deemed the best performer according to the page Combinations report. Of course this is after you finish running your head-to-head follow-up test between your control and the best performing page combination to verify your results outside of the multivariate testing environment!
Google Analytics App for the iPhone – A Quick Review
Posted by: | Comments
The best Google Analytics App available for the iPhone right now is in my opinion without a doubt (and simply named) Analytics App.
Analytics App gives you complete mobile access to your Google Analytics data from your iPhone. Whether you have just one site profile and one Google Analytics (GA) account, or multiple accounts with multiple profiles, this iPhone app is something I believe you will really be happy with.
For the past two months I have been using Analytics App at least a couple times per day, be it to answer questions on the go for others (i.e. HiPPOS) or even just simply playing around with it for fun to satisfy curiosities. It’s intuitively easy to use with virtually no learning curve for the app itself. Basically just click and there’s your data. I wouldn’t use this for actual data analysis (and this in part because of the limitations of the iPhone itself) but it’s extremely handy and fun to be able to have on the go with you.
Before I purchased it though, I thought it might be difficult to view analytics data on the iPhone due to the screen size, but the interface fits the iPhone perfectly and the data surprisingly is extremely easy to view with no straining on the eyes, even the charts it displays are easy to view.
More than 40 Reports Are Available to You
Analytics App has 9 standard Google Analytics Overview Reports:
- Dashboard, Visitors Overview
- Traffic Overview
- Content Overview
- Event Tracking Overview
- E-Commerce Overview
- Site Search Overview
- Goals Overview
- Quick access Dashboard for Today’s data.
Plus, more than 30 detailed reports you can view by drilling down in each of the areas, for example Keywords, Visitor Loyalty, Top Landing Pages, Top Content, Event Actions, and the ability to view your Custom Reports that have already set up inside your Google Analytics account.
Unfortunately, and unless I am missing something, you can’t click on individual referring sites themselves to view them in the iPhone browser. And, URL’s are limited to 22 characters so in the Top Content Reports you’re not able see the entire URLS or click on them either, so if your URL’s are long and non-intuitive before the 22 characters are used up you may not be able to know which URL is being referred to (but you can always use the Content by Title report).
Viewing Multiple Accounts
I should note, that to view multiple accounts with Analytics App you’ll need to give access for one account to view the others – but this has to be done on GA website itself, or you do it the old fashioned way by logging-out and then log back in on the app for each account itself if you want to view different accounts (I want to emphasize this is for accounts, not profiles as all profiles under your account are visible and selectable when you launch the app).
Misc.
Security: Analytics App stores your GA username and password information locally on your iPhone for security reasons.
Date Ranges: I found that setting the dates to view your data for is actually easier than in Google Analytics using the standard iPhone scroll wheel (and setting it in GA is easy).
Cost
What impressed me the most before I purchased it was that not that it was only $5.99, or that its reviews were overall rather fantastic, but that they offered a money back guarantee via PayPal so I felt that the risk was minimal. I’ve purchased many apps that have left me disappointed, so this was a motivating factor to purchase it on the spot without further consideration.
In Conclusion
So with nothing to lose (your money that is), I would definitely recommend checking the Analytics App out, or if you know someone who already has it to try theirs out – you’ll be glad you did, the app itself is really well produced.
Properly Setting Up Excluded Placements Inside Google AdWords
Posted by: | CommentsDo you advertise with Google AdWords? Do you have your Google Adwords paid search ads show up on their Content Network? Not sure what that exactly means?
Google allows you to not only display your ads in Google’s search results pages when someone performs a search on Google, but also on sites that use their Google AdSense program to automatically display paid search ads that are relevant to their website copy – although many times it’s not always relevant – but which makes this post relevant to you!
You can set up lists of sites or domains that you do not want your Google AdWords ads to show up on via the Excluded Placements section (found in the Networks tab of the new Google Adwords interface). By viewing the detailed stats of your ads on these sites, assuming you have your conversion tagging set up to show AdWords conversions correctly (surprisingly many put it on the wrong page) you can determine if your ads that are appearing on specific sites on the content network are profitable or not for you.
If not set-up properly though, you may be surprised to find out that your ads are still appearing on sites that you thought you had banned through the Excluded Placements section in the AdWords interface.
According to ROI Revolution’s Make Sure Your Excluded Placements Are Actually Being Excluded post, there is a easy fix in the Excluded Placements section to make sure you are correctly banning an entire site from having your ad appear on it.
Unless you specifically want to ban just a certain subdomain such as www.about.com, or marketing.about.com, but want your ads to appear on other subdomains of their website, you need to enter in just about.com. Entering in the root domain (about.com) without the third level (www, marketing, etc.)you will be successfully banning the entire domain not just that specific third level entered.
For more detailed information, you can view Google AdWords help section on setting up Excluded Placements.
When is Testing for Conversion Optimization not the Right Thing to Do?
Posted by: | CommentsThere are just certain times when running a multivariate test to optimize web page conversions will produce unreliable results. Results that either will not yield statistically significant outcomes, or outcomes that even though the numbers may show statistical significance at the end of your test, would not be reliable enough to roll-out and see the nearly the same results much longer than after that particular testing period ends. Remember you are looking to take one step forward and improve your web pages conversion, and not two steps back rolling-out a page that ultimately performs worse than your control; which is quite possible if you are not mindful of certain instances.
Such instances include:
Seasonal traffic – Testing pages during specific high seasonal times for your business although may produce statistically significant outcomes by looking at the numbers themselves; the changes made based on the test outcomes would not be reliable after the seasonal traffic ends. The user intent during these times in most cases is not typical user intent or behavior displayed during the non-seasonal times, and in some cases also between seasonal times.
Traffic sources that fluctuate in delivery volume -If viewed at in a line chart would show high peaks and or low valleys (or may even show times of nonexistence traffic). This traffic volume is too unstable and therefore an unreliable indicator of ongoing performance. A specific instance would be running a multivariate test on a landing page that the traffic delivered to the test is from various different email campaigns. Also be careful of a test that suddenly receives a spike in traffic due to a current event for example that would send a large volume of traffic of non-typical visitors into your test.
Low traffic volumes – if your page does not receive enough consistent traffic of a certain volume than the likelihood of high confidence statistically significant results is slim-to-none in most cases. You need to have enough traffic to produce enough conversions (a conversion being anything you deem to be one, from a registration, to even a download) that your results will be accurate. Many conversion optimization experts say at least 10 conversions per day is the absolute minimum needed to run a test.
And if you’re A/B testing:
When you can’t run your control in unison with your test panels – without simultaneously running your control panel along with your test panels you will not be able to accurately assess the results of your test. You need to be able to assess how each of your panels or page combinations, both control and test panels, perform under the identical conditions and time period. The only way to accurately do so is to have them run simultaneously with your traffic randomly split amongst them.
