Wednesday, October 3, 2007
Moscone Center West Hall
San Francisco, CA

Join us for the Optimization Summit in San Francisco, CA on October 3, 2007 and discover the multivariable optimization strategies that make our clients so successful.

You'll Also:

• Learn how companies like Delta Air Lines and AOL are achieving major successes online using multivariable optimization strategies
• Hear Anne Holland, President and Publisher of MarketingSherpa, give a keynote on becoming a test-centric organization
• Select the session track option that addresses issues relevant to your unique business needs

Visit www.optimizationsummit.com to learn more.

Don't Miss Out! Early bird registration prices end on September 15th!

Optimost Webinar Series

The Optimost Webinar Series consists of two monthly presentations of how-to seminars, real-world case studies and tips on how to increase customer conversion rates online. Webinar sessions alternate between general sessions open to the public and exclusive sessions only for Optimost customers.

To Gain Confidence, Check Your Sample Size
With Guest Speaker Sunil Gupta, Ph.D.
(General Webinar)

Wednesday, September 12, 2007 – 1:00pm ET (10:00am PT)

Among the first questions asked by novice testers are, “How do I know my sample size is big enough?” and “What can I do to gain more statistical confidence?” While these questions are important, it is more useful to understand the statistical bases for properly designing and executing tests and properly interpreting their results.

In this webinar you will learn:

• How to determine the correct sample size so you can be sufficiently confident when testing differences between various outcome measures, e.g., click-through rates, dollars per sale, downloads per visit, etc. Simple calculators will also be provided.
• The relation between sample sizes for A/B tests and those for multivariate tests
• How to design a proper test to assure representativeness and proper randomization
• How to detect flaws in test execution (from things beyond your control), and how to “fix” them statistically
• How to properly interpret test results and then determine what to test next

Register Today

How to Use Clickthru Groups
(Customer Only Webinar)

Wednesday, September 26, 2007 – 1:00pm ET (10:00am PT)

This customer only webinar will focus on the use of clickthrus and clickthru groups during an experiment. Even when not used as the primary success metric in your experiment, clickthrus can be still be utilized to give you a deeper insight on how your users are behaving on your site. Setting up clickthru groups in your reports can give you granular information, not only on areas on your page that are helping your success metric, but also on areas that might be cannibalizing your conversions. This webinar will focus on the benefits of clickthru groups and provide some great best practices to get you thinking how you can best optimize your page.

Register Today

Customer Success Story

Established in 2003 to offer luxury, fresh flowers and UK next day delivery service at reasonable prices, Serenata Flowers has grown rapidly to become the UK's third largest online florist. As a purely online vendor, Serenata adopts best of breed web analytics, email marketing and optimization services to drive their business aggressively forward.

Optimost currently partners with Serenata Flowers to conduct ongoing multivariable tests and persona targeting on all pages in the sales funnel including, landing, home, category, product and checkout pages. Using the Optimost platform to rapidly test multiple permutations of layout, copy and design of home page offers, Serenata Flowers was able to identify that visitors to their site prefer offers that highlight a specific style of benefit messaging, brand identity and graphics. Analysis of the dataset captured by the Optimost platform enabled Serenata Flowers to deliver a customer driven experience that ultimately improved conversion by 27%.

Key findings include:

• Buttons matter, but more so the text or call to action than the color
• Headlines matter, and the wording or underlining of a headline makes a difference
• Risk reversal messages work, such as “100% satisfaction” or “you can always revise your offer later”
• Messages focusing on benefits work, such as “free delivery” or “free offer”

Using Optimost for testing and persona targeting enables Serenata Flowers to focus on its strategic growth areas of their business and continue its success.

Ask Dr. Montero

Each Month, Dr. Michael Montero, one of our experts in statistics and experimental design, addresses your testing-related questions. Here is this month's question:

When optimizing revenue per visitor, how do we calculate the expected traffic needed for reaching statistical significance?

Conversion rates and clickthru rates are common metrics used for web page optimization. Revenue per visitor is another very important metric that can be measured and maximized through multivariable testing. In the case of a retail website, conversion rate optimization will tell you what percentage of your visitors are successful purchasers whereas revenue per visitor goes a step further by telling you on average how much money each visitor is spending. Traffic projections for revenue per visitor are made in a similar way as conversion rates (see January 2007 newsletter), but there are some slight differences.

Let's look at an example to figure out how long a test may need to run before a particular comparison has reached significance. We'll assume that for a promotional ad being tested, there are two variations: “ad present” (Value 1) and “ad not present” (Value 2). In order to predict the duration of the test or how many visitors will be needed to reach statistical significance, there are three pieces of information required:

• Expected impact or difference between the two values being tested
• Confidence level (associated to a z-score value)
• Standard deviation of the revenue (STDEV)

In our example, the current revenue per visitor to the test page is $10.90/visitor and we would like to raise this to $11.45, an impact of $0.55 or a 5.0% lift. Next, a 95% confidence level is chosen as the criteria for reaching statistical significance. Finally, the standard deviation of the revenue can be obtained by current web analytics monitoring this page just as we obtained the current revenue per visitor. It's very important that the summarized statistics for current revenue per visitor and standard deviation of the revenue be based off of data that do not contain atypical or high purchases. Such outliers will skew the prediction by inflating the revenue per visitor number and standard deviation. In this case, let's assume the standard deviation is $40.00. The standard deviation of the revenue will give you an idea of the fluctuation of revenue relative to the mean.

Equation 1 shows the formula for calculating the z-score based on the difference of two means and their respective “pooled” standard error. Since we are making a projection, we will assume that both means will approximately have the same standard deviation (STDEV1 and STDEV2) and sample size or number of visitors (N1 and N2). Therefore STDEV = STDEV1 = STDEV2 and N = N1 = N2 (See Equation 2).

Next, the number of visitors for each value, N, is solved for and shown in Equation 3.

In this example, we've chosen a confidence of 95% which corresponds to a z-score value of 1.96 (see the January 2007 newsletter for how this is calculated). Equation 4 shows the final calculation.

Therefore for this test, we would need 40,637 visitors per value, or a total of 81,274 Visitors. Given that this page sees 10,000 Visitors per day, we would expect 81,274/10,000 = 8.1 days for this test to run.

If instead the objective of the test was to optimize items per visitor, Equation 3 can also be used whereby you would need the expected impact for items per visitor and the standard deviation for items. In general, Equation 3 can be used when dealing with non-binomial data (data that is not simply success or failures such as conversion or clickthru rates) where the measurements of the metric can have more outcomes.

If you would like to submit a question to Dr. Montero, please send it via email to newsletter@optimost.com.

Partner Spotlight

Delivering Relevant Content with Deep Customer Insight
Readers of this newsletter already know that consumers are in control of their online experiences. They demand that web sites deliver exactly what they're looking for. When you listen to — and learn from — your customers, you gain actionable marketing intelligence. This lets you develop relevant, engaging content that speaks directly to the right visitors at the right time.

Optimost has formed a strategic partnership with WebTrends to help you understand the intent behind each customer visit to your site so you can seamlessly deliver targeted, optimized user experiences.

WebTrends is a leading provider of web analytics and consumer-centric marketing intelligence solutions. While most people are familiar with their web analytics products, they've recently introduced a new version of WebTrends Marketing Lab, their complete solutions suite, to put the consumer at the center of the marketing universe.

• Learn more about the solutions suite
• Watch the ML2 webcast

With the integration of WebTrends and Optimost, you can accurately segment visitors based on behavior and easily adjust web content and campaigns, ultimately delivering what's most relevant for keeping people engaged. You can:

• Automatically turn visitor intelligence into action and results
• Optimize the value and strength of customer relationships over time
• Improve top-line business performance through a proven conversion optimization methodology
• Focus on strategic marketing — not just tactical execution

WebTrends will be featured at our upcoming Optimization Summit, and Optimost will be taking part in the WebTrends Engage'07 Conference. To learn more about WebTrends, visit their site at www.webtrends.com, or call 1-877-932-8736.

Click here for a complete listing of Optimost Partners.

If you are an Optimost partner and you are interested in either having your company featured in our Partner Spotlight or your upcoming events listed in our newsletter, please send an email to partners@optimost.com.

In The News

Optimost Takes Optimization Technology to New Heights — Providing Enhanced Search Engine Segmentation and Mobile Capabilities

In August, Optimost announced new features which include enhanced audience tracking and search engine targeting. Optimost also released new functionality that provides for automatic tracking and targeting by browser, operating system, and type of device — including mobile phones such as the new Apple iPhone and gaming consoles like the Nintendo Wii. Read more about the new features on our website.

Mark Wachen Interview on “Online Marketing with RSS Ray” Radio Show

Mark Wachen, the CEO of Optimost, will be interviewed on the “Online Marketing with RSS Ray” Radio Show. The interview topic will be “Focusing on Multivariable Optimization” and will take place on Wednesday, September 19, 2007 at 1pm ET/10am PT.

To learn more about the show and find out how you can listen to the interview, please visit the RSS Ray website at www.rssray.com.

Recent Articles

Little Things Mean Much On Web Sites
Investor's Business Daily, August 15, 2007

Slightly Tweaking Controls Helped In Lifting Delta.com
CNN Money, August 15, 2007

Optimost Releases Mobile Tracking and Targeting Tool
DM News, August 28, 2007

If you are interested in learning more about Optimost or would like to meet with an Optimost representative, please feel free to contact us.

Optimization  •  Technology  •  Services
Clients  •  Resources  •  Company
Privacy Policy  •  Legal Notices