It’s imperative to document every effort of your online testing and optimization program. Not only to see the progression of improvement over time, but to also have reference for future tests you are planning or questions that may arise from others you are working with.

How I document my A/B and multivariate testing is as follows:

First, I primarily use Excel for all my documentation efforts due its ease of use.

I create a Master Test spreadsheet that serves as a very top-level summary for quick glancing. This spreadsheet is constructed to have column headers for the following:

• Test name – giving each test a descriptive and unique name
• Test date range – documenting the start and finish date of my tests
• Test hypothesis – why I am running this test and why I believe my outcome will be such
• Results – brief description of the result

Each row is for a single test. I then link the Test Name cell for each test to another spreadsheet that is built specifically for that test (If you have ten tests in the Master Test spreadsheet each test would link to its own Individual Test spreadsheet for a total of 10 spreadsheets plus the Master Test Sheet).

Each Individual Test sheet contains multiple tabs and information, and this is where the detail will go throughout the test.

The first tab contains the test information and summary and broken into sections:

• Test name
• Test date range
• Test hypothesis
• What type of test (a/b, multivariate), and how many panels or combinations
• Traffic data (source of traffic, and current traffic stats)
• What type of metrics I will be using to determine the results and how to determine the metrics (is conversion impressions divided by sales, or impressions divided by clicks on a certain button etc?)
• A space to record final metric results (control performed as such, top performers identified individual performed as such)
• Learning’s (both as the test is live and from the results)
• Ideas for future tests based off of this information
• Next actions (will this be rolled out etc.)
• Miscellaneous notes

I also have other tabs in the Individual Test spreadsheet:

• Screenshot of control
• Screenshots of test panels or combinations (depending on how many there are). If there are too many panels or page combinations to take screenshots of, after the test is ended I take screenshots of the top performing test panels for future comparisons)
• Screenshots of Test statistics (When I am using Google Website Optimizer I take daily screenshots of the stats admins. and store in a separate folder, but the final screenshot from the point at which we end the test is stored in the Individual Test spreadsheet – just in case I transcribe something wrong I have an actual reference to go back to.
• Various other tabs as necessary for reference such as more detailed metrics information, etc.

I also keep a folder for each test (with the folder using the test name) that contains my test spec PowerPoint so that I can see all of the elements or options that we are using, analytics data, screenshots of everything-basically anything used from the conception of the test  all the way through to the end.

What this enables me to do is at any point in time have a huge history of each test both from a visual standpoint and data-driven standpoint. The Master Test sheet gives me quick access to the individual tests but also a timeline of the testing I have done.

My Testing Documentation Flow