Ah, summer. Fourth of July fireworks. Walks on the beach. The hot, humid heat rising from the city streets. It's the perfect time to start thinking about Christmas.

Why Christmas? Well, here in the heat of summer, it's a great time to start a strategic optimization program so that you have time to get your website in tip-top condition for when the holiday rush begins later this year.

Some of our customers struggle with this idea. Their first reaction is: how can I optimize for the holidays when I know that my popular products will change? No one wants to buy a tank top in December. And my holiday specials are still just a twinkle in my promotion manager's eye.

But an optimization program is a lot more than just offer testing. There are plenty of other variables that can be optimized today and still be very relevant when the holiday season is in full swing. For example, improvements made to pages downstream in a funnel prove to be relevant all-year round. What is the optimal order of presenting fields in my form? What should my submit buttons look like? How detailed do the instructions need to be on the shipping page?

On a landing page as well, many key questions can start to be explored now so that the optimal answers are in place when the holidays arrive. How prominent should my search box be? Should I add extra navigation areas to the bottom of the page? Would increasing type size improve response?

Another important benefit to starting your strategic optimization program now is that you can start to understand how different audience personas behave when they visit your site. Optimizing a website is more than just optimizing for the audience as a whole. It's important to understand if the behaviors of different persona groups vary when they visit your site. For example, the behavior of people coming in directly from a paid search ad is very different than people coming from an ad banner. The behavior of new visitors to a site is very different than repeat visitors. People coming to the site during the workday may be very different from people coming at night. By starting to track and optimize now, you can start to understand these differences and start optimizing your site for these different audiences. Furthermore, since the concentration of visitors from each of these persona groups may change in the holiday season, you'll already know the best solution for the groups that may be critically important then, but less numerous today.

A final important reason to start your optimization program today is to insulate yourself from the marketer's fourth quarter nightmare: the dreaded code-freeze. Many sites are reluctant to make changes to their code right at the time when optimization is most crucial, the holiday season where traffic increases substantially. So now is a good time to get your technical team onboard, and make the necessary code changes required so that you as the marketer can still be optimizing even when you are not allowed to go in and modify the code in December. By doing so, you'll be able to continue to test the variables that potentially take on greater importance when the holiday crowds start arriving. How important is 24-hour shipping? How prominent should my return policy be? How important is gift wrapping?

So the next time you hear “Summer in the City,” “In the Summertime,” or “Sunny Afternoon,” don't just think about swimming in the sea or sipping an ice cold beer. These summer anthems are also the cue for starting your holiday strategic optimization program.

TrueCredit develops and markets credit-information based services that deliver value to both consumers and the nation's largest financial institutions. The company goes beyond the delivery of credit data with a unique product set that is designed to help consumers manage their debts as efficiently as they do their investment portfolios.

TrueCredit wanted to increase the number of users who moved through the sales funnel, and brought in Optimost to optimize their home page. The dedicated Optimost Client Team developed a test plan for the page, which included 10 variables and 64 different values (versions of variables) to be tested. The Optimost solution engine quickly identified and generated over 68,000,000 template permutations. Optimost conducted multivariable tests on the TrueCredit home page to identify a winning creative that increased subscriber confirmation by 10%.

Optimost determined that the TrueCredit home page had increased conversions from several simple changes, which included:

• Replacing the main image
• Adding cancellation language to the page
• Removing pricing information
• Moving the location of the submit button

The resulting improvement made a positive impact on TrueCredit's business, and the company has used this first step to then launch optimization of pages down the conversion funnel to reduce further abandonment and grow their business.

How do we ensure that multiple pages being tested do not perform better than others simply by luck?
When testing more than two pages (or making more than one comparison), the probability of observing a false effect increases. Typically, a 95% confidence is used as criteria for assessing a comparison's significance. For example, if we're testing two different headline copies, and we can calculate that Page B is converting better than Page A with 95% confidence, we can state that B's improvement over A is statistically significant. However, let's take another scenario where eight pages are being tested (A through H different headlines) and one of the pages appears to be performing better than the benchmark page at 96% confidence. As the number of independent comparisons increases in an experiment, there exists a higher chance that something will appear significant for no reason (a “false positive” or Type I error). Figure 1 depicts the two types of comparisons. Why does the probability of a false positive increase with the number of tests?

Figure 1. Given 95% criteria for significance, (a) one comparison test has a 5% chance of detecting a significant difference whereas (b) seven comparisons have a 30.2% chance of detecting a significant difference

For a 95% confidence level given the single comparison in Figure 1(a), we expect to observe statistical significance, purely by chance, 5% or less of the time. The flipside of this is to say that 95% of the time, we should expect no difference between the two. Now, let's say that two comparisons (or three pages are tested) are being made: A vs. B and A vs. C. If the two comparisons are independent from one another and we rely on a 95% confidence level for significance, then the probability of expecting no difference for both comparisons is 0.95 x 0.95 = 0.9025 or 90.25%. The flipside of this is saying a significant difference may be noticed at 9.75% or less of the time. Basic probability theory simply allows the multiplication of the two probabilities of each event in order to compute the overall probability of both events occurring at the same time (See equations 1 and 2).

For multiple comparisons, this application of probability theory can be extended. In the case for Figure 1(b), we essentially would multiply 0.95 by itself seven times (0.95^7) to calculate the probability of all seven comparisons (eight pages being tested with seven being evaluated versus the benchmark) having a null effect which is 69.8%. Therefore, the chance of detecting a significant effect is 1 - 0.698 = 0.302, or 30.2%. As you can see, there is a greater chance that something may be considered significant simply due to a higher number of pages being tested and hence the likelihood of a false positive occurring increases.

How can we better discern between true effects and false positives? We do this by tightening our criteria for significance. One way to do this is using a very simple correction method developed by an Italian mathematician, Carlo Bonferroni. Bonferroni's correction takes the original level of significance and adjusts it based on the number of comparisons being made. For example, when making seven comparisons with a goal of 95% confidence, the Bonferroni correction raises the required level of significance for any one comparison to 99.29% confidence. Thus, going back to the original example in Figure 1(b), a page performing at an unadjusted 96% confidence is not considered significant since it is not higher than the adjusted confidence of 99.29%.

The Bonferroni correction is one of a number of mathematical methods that can be used for adjusting significance levels for multiple comparisons. The following are other safeguards that can be used to help avoid a potential false positive:

• Continue gathering sample and monitor sustainability of margin of improvement
• Evaluate the marketing content for the test page to see if the associated confidence and lift is reasonable
• Retest the page
• Move to a multivariable testing approach and include the variation in question in the experiment

The last point is the most important since the multivariable testing approach will better provide a more robust environment for the variable being tested. In the example above, the headline would be included amongst other variables on a page being tested. If the same improvement is detected by a specific headline, it would have been tested with other variations on the page making its effect more robust than simply being tested in an A/B fashion. False positives may occur on a variable level within a multivariable approach, and the same Bonferroni correction can be applied. It is always best to retest a questionable value and tighten the confidence level especially if the content of the value provides an unreasonable margin of improvement given its content.

As we have mentioned several times in this newsletter, smart marketers are looking to solve the “X times Y” equation, where X is getting the maximum number of visitors to your site for least amount of marketing dollars, and Y is maximizing the percentage of those visitors who become customers. Optimost clients are maximizing the conversion rates on their websites (the “Y” part of that equation), and Optimost partners with a number of companies that focus on “X”, getting those visitors to your site in the first place. One of the most popular ways to generate web traffic is through a comprehensive email marketing program.

GOT Corporation's Campaigner solutions allow organizations to quickly implement effective and relevant permission-based direct marketing campaigns. Campaigner products enable organizations to communicate with customers, measure how they respond, and analyze those responses to interact in a more intelligent, automated way — resulting in more profitable relationships.

In order to accommodate diverse marketing needs, GOT Corporation offers two email marketing solutions: Campaigner and CampaignerPro.

Campaigner: Fast, easy and affordable Email Marketing. Campaigner's self-serve tools help you create professional email campaigns in minutes, and build and manage permission-based contact lists with ease.

CampaignerPro: Powerful one-to-one marketing the delivers targeted, relevant information to your customers & contacts. CampaignerPro's advanced features — segmentation, dynamic content, recurring scheduling — put intelligent marketing automation to work for you.

For more information about GOT's Corporation's solutions, please go to www.campaigner.com.

Click here for a complete listing of Optimost Partners.

If you are an Optimost partner and you are interested in either having your company featured in our Partner Spotlight or your upcoming events listed in our newsletter, please send an email to partners@optimost.com.

Optimost is partnering with the Online Marketing Summit series to speak at the ten cities included on this year's tour. The Online Marketing Summit is an exclusive set of events for leading marketers designed to share interactive marketing best practices while further educating participants on the critical marketing topics in today's economy.

Below is the list of dates and locations for Online Marketing Summit:

• July 10, 2007 - Atlanta, GA
• July 12, 2007 - Washington DC (Falls Church, VA)
• July 17, 2007 - New York, NY
• July 19, 2007 - Boston, MA
• July 24, 2007 - Denver, CO
• July 26, 2007 - Houston, TX
• July 31, 2007 - Milwaukee (Brookfield, WI)
• August 2, 2007 - Chicago, IL
• August 7, 2007 - San Francisco, CA
• August 9, 2007 - Los Angeles (Long Beach, CA)

Click here to save $150 off the regular price and to register for the event.

To ensure customers are updated with the most current information on our Campaign Management Console (CMC), Optimost will begin distributing quarterly alerts to customers with detailed information The first quarterly alert is scheduled to go out mid-July. Be sure to keep a lookout for the first installment!