Stop by our booth if you plan to attend these events, and if you’d like to schedule a meeting with an Optimost representative in advance, please

CyberMonday did not have a great 2006. From a sales standpoint it did just fine. From every indication the 2006 online holiday season is off to another record breaking start. comScore reported that sales on so-called CyberMonday (the first Monday after Thanksgiving) were 26% higher than in 2005.

But the concept of CyberMonday took a beating. "CyberMonday not all it's cracked up to be," Marketwatch complained. "E-Looking but Little E-Buying on Monday," grumbled the New York Times. And News.com gave the ultimate slap in the face, "CyberMonday, You're No Black Friday".

It seems that an expectation had been set that CyberMonday represented the biggest online sales day of the year. However, when many of the ecommerce scorekeepers re-checked their 2005 statistics, they discovered that CyberMonday was not at all the top day. It wasn't even in the top five. The media picked up on this recently and as evidenced by the aforementioned articles, decided to slam CyberMonday six ways to Sunday.

In truth, this wasn't really fair. If one looks back at the Shop.org press release that birthed CyberMonday, it's clear that it was never proclaimed that CyberMonday was the biggest sales day, but "simply one of the biggest online shopping days." The idea was that CyberMonday was to mark the beginning of the holiday rush - not the peak. And the facts seemed to bear this out, as traffic does start to begin its end-of-year rise right after Thanksgiving. But clearly, something got lost in translation.

So if CyberMonday is not the king of hill, what day is? On this, there seems to be some disagreement. Based on 2005 data, MasterCard said it was December 5 (the second Monday after Thanksgiving). Paypal said it was December 12 (the second-to-last Monday before Christmas). Will the real CyberMonday please stand up?

In all likelihood, the reason why no one can agree on this is because there probably is no right answer. It's important to remember that these are aggregate numbers across a large variety of websites. These statistics should include a caveat with that old standby, "individual results may vary," because the truth is that if you look at the data from an individual retailer standpoint, there is significant variability. When we looked at the 2005 data for our clients on a site-by-site basis, we saw a wide variety of CyberMondays (indeed, none of them proved to be the mythical Monday-after-Thanksgiving CyberMonday). December 5th did pop up, and so did December 12th. But we also saw others, including one as late as December 19th. (This one seems to deserve its own moniker. Perhaps "Procrastinator’s Payday?" "Oh-crap-I-forgot-to-by-my-wife-a-Christmas-present-
and-I'm-too-busy-to-run-to-the-store Monday?").

This is critical information to know because if your product is one that appeals to the procrastinator in many of us, you still have time to be testing and fine-tuning for your big day, which this year could likely be December 18th or even later. In fact, the week, or even weeks, after Thanksgiving may be the ideal time to do some final testing because you can leverage the higher traffic that the mythical CyberMonday does bring, allowing you to test more options more quickly.

So the moral of the story is that yes, there is a CyberMonday, but on an individual basis, it comes in many shapes and sizes. The important thing is to make sure that you are accurately tracking your own site's behavior, so you know when your CyberMonday actually falls. That way, you'll be able to fully capitalize on what surely is the most wonderful (e-commerce) time of the year.

Time-Life is one of the world’s pre-eminent creators and direct marketers of unique books, music, videos/DVDs, and multimedia products. Among the wide variety of music categories on its website (www.timelife.com) is a segment called “70s Music Explosion.” A recent analysis of the product page for this segment revealed that few visitors were making purchases or adding products from this page to their shopping carts. In addition, the large amount of information on the page, coupled with its complex DHTML scripting, made it difficult to analyze, test, or even modify.

Time-Life turned to Optimost to help them optimize and improve the performance of the 70s product page. The dedicated Optimost project team worked with Time-Life to develop a test plan which included 10 variables and 63 different values (versions of the variables). Based on this test plan, the Optimost solution engine quickly identified and generated over 1 billion possible permutations of the page.

The test was then conducted on a subset of these possible permutations, and as a result, Optimost identified a winning creative with a UAR (Unique Action Rate) increase of 75% for customers adding items to their shopping cart, and put Time-Life on a pace for a seven-figure increase in annual online sales.

For more details on this success story and other customer success stories, please see http://www.optimost.com/case_studies.phtml.

What is a confidence interval and what factors affect the size of the interval?
A confidence interval (CI) represents the range in which a result will fall based on a desired probability. When optimizing a web page, the conversion rate for that page will have a mean (average) and an associated error for that mean. Given the error, as well as the conversion rate, the number of visitors to the page, and the desired probability, a confidence interval can be constructed. For example, the following 95% confidence interval [9.675%,10.825%] indicates that the page being tested has a 95% probability that its average conversion rate will appear within the range of 9.675% to 10.825%. The interval can also be interpreted as stating that there exists a 5% probability that the average conversion rate will not appear in this interval.

Error results from a variety of different sources outside of the variables being controlled by the experiment. In this example, the average conversion rate may be 10.25% at the onset of the experiment, but toward the end of the experiment the rate is observed to be 10.45%. This minor shift can be attributed to slight changes in the audience, temporal effects (weekday versus weekend, daytime versus nighttime, etc.), dynamic data (rotating ads, product data, news info, etc.) not controlled by the test, latent conversions due to consumer decision making and other uncontrolled influences. Error can be quantified in a number of different ways and is typically represented by a standard deviation or standard error. The following terms make up the necessary components for calculating a confidence interval for that conversion rate:

N	=	number of visitors to the page being tested
p	=	the conversion rate
SE	=	standard error = standard deviation of the sample of means
z	=	statistic value or score determined by the desired confidence required

The z-score can take on different values given the degree of certainty required for the interval. For a two-sided interval (meaning the error can be greater or less than the mean), a 95% confidence would require a z-score of 1.96, and a 99% confidence interval would require a z-score of 2.58. The CI can then be calculated using the following formula:

Therefore:

From these formulas, one can see that the standard error or the z-score can impact the width of the interval. If we decide on a 95% confidence, then the only way the interval can increase or decrease is by the changes in the standard error (which is in turn determined by the conversion rate and the number of visitors).

Ideally, the confidence intervals should be as narrow as possible so that discernment between the conversion rates of pages being tested is clear, having little overlap between their respective intervals. In Figures 1 and 2, we see two versions of a page, A and B, converting at approximately 8.5% and 8.0% respectively. Page A is showing a 6.3% lift above page B (or +0.5% impact). Figure 1 shows relatively large confidence intervals for both pages which are caused by the low number of visitors which have reached these pages. Such large intervals are not uncommon at the onset of an experiment but with more time (and hence more sample), these intervals will reduce in size. As the experiment continues to accrue more visitors, the intervals will begin to resemble those in Figure 2. With an increase in the visitor sample size and sustained lift, the greater the certainty that page A will continue to outperform page B.

Figure 1. Confidence Intervals for Page A & Page B with Small Sample size	Figure 2. Confidence Intervals for Page A & Page B with Large Sample size

If you would like to submit a question to Dr. Montero, please send it via e-mail to .

Last month, we began highlighting our partners in the web analytics space. Optimost partners with most of the leading web analytics vendors, as Optimost gives you a platform to address bottlenecks in your website that are made apparent by your web analytics package. This month we highlight Optimost partner Omniture.

The Omniture Online Business Optimization Platform provides customers with the ability to collect, integrate, analyze, and automate the reporting and distribution of multi-channel business information and processes. With Omniture, large volumes of data generated by Web sites and other business systems can be captured, stored and analyzed to:

• Measure trends and customer behavior in real-time
• Provide real-time, high-performance analysis and reporting for all levels of business users
• Automate new online processes
• Optimize overall business performance

For more information about Omniture, go to www.omniture.com.

Click here for a complete listing of Optimost Partners.

If you are an Optimost partner and you are interested in either having your company featured in our Partner spotlight or your upcoming events listed in our newsletter, please send an e-mail to .