| For any sports fan, these hours of debate are one of the great
joys in life, one of the great ways to pass the time when stuck
in a traffic jam, or waiting in a restaurant for a long overdue
meal. But in the conference rooms of Web-based companies, we are
subject to other great debates, and these can be much more disruptive.
Should our registration process be two short pages or one long one?
Should we use AJAX or static HTML? Should we have red submit buttons
or green ones? (Compared to the Nastia vs. Nadia debate, one must
admit that these debates do sound a tad pathetic ?.)
The amount of time spent on these types of debates can be tremendous.
We confirmed this during a recent poll we conducted during an American
Marketing Association Webcast on “How
to Create Powerful Web Copy That Sells”.When asked
what is the primary way that decisions are made in your company
about what goes on a Website, fully 39% of the attendees responded,
“A group sits around a conference room table to debate/argue
about it.” When you consider that these debates often can
involve 5–10 people spending one or two hours in heated debate
getting frustrated with each other, it becomes clear that the opportunity
cost can be tremendous.
In addition to the time wasted, the impact on morale can be significant.
Inevitably in these debates, the HiPPOs (the “highest paid
person’s opinion”) or what I’ll call the LIONS
(“Loud, Irritating, Obnoxious Naysayers”) hold sway.
In fact, our poll also found that in 27% of cases, the primary way
decisions are made is by deferring to senior management. The quiet,
analytic, insightful employee (the Quail?) doesn’t stand a
chance in this conference room jungle. Many will conclude that the
effort to get their truly insightful idea across is just not worth
the emotional effort.
The tragedy of this dynamic is that unlike the great intergenerational
sports debates in which the answer can never be proven, these Web
design debates can easily (and efficiently) be answered through
multivariable testing. Instead of arguing in a conference room,
the great battle between static HTML vs. AJAX can be played out
in real time as customers visit your site and vote with their mouse.
One of the great virtues of multivariable testing and optimization
that we like to point to is the measurability. Since multivariable
optimization always involves running a control version along with
the test versions, you can determine with pinpoint precision the
value gained from a given optimization project. The hard dollar
benefits can be substantial, but often overlooked is the “soft
dollar” benefit of testing. We often hear from customers that
one of the great unexpected values they get from a testing program
is the time they save arguing with one another. As Abby Stephenson,
the Usability Manager of Delta Air Lines, aptly put it, “We
used to have long debates about the minute details of a page. Now,
we are able to move much more quickly knowing that we will test
it after we go to production.”
In fact, some of our customers take it even one step further. We’ve
seen customers set up office pools in which employees can bet on
what will win in a current optimization project. So in addition
to the annual Academy Award pool or NCAA March Madness brackets,
they’ve added a monthly optimization tournament to see, scientifically,
which employee knows the customers best. In effect, they’ve
transformed the morale-draining process of conference room squabbling
into a morale-boosting exercise.
You’ll be amazed at how empowering instituting an optimization
program can be for everyone in the office (except for perhaps the
HiPPOs and the LIONs). You’ll be amazed how much time is freed
up. Instead of arguing over the red vs. green submit button, you’ll
have much more time to tackle much more important strategic discussions
like brainstorming the next great product idea or analyzing the
profitability of free shipping. And as a side benefit, you’ll
have much more time to spend on the truly important debates: Michael
Phelps or Carl Lewis?
|