| If there is a lesson in all of this, it is that even for the
experts, predicting consumer behavior is tough. This message rang
loud and clear during an interesting contest that we conducted at
Interwoven's GearUp conference this past April. During the event,
we showed our audience five variables that were tested in a multivariable
optimization project we conducted on a signup page for Sherman's
Travel. The variables included an image, a background color, a submit
button design, and several others. For each of the variables, we
showed the audience several of the different values that were tested,
and asked everyone to vote on what was the winning value when actually
tested against Sherman's Travel's live traffic.
The results were quite stunning. Not one of the hundreds of attendees
who participated in the contest guessed all five winning values
correctly. In fact, the most any attendee got correct was three.
And fully 42% of our audience, an audience that was packed with
thought leaders across the interactive space, got none of the answers
correct. The average number of correct answers by our attendees
was 1.07. If you had a dog pick the answers at random, the expected
score would be 1.28.
So what does this mean? Does the average expert interactive marketer
provide no more value than the average dog? Does it mean that there
is no role for an expert in interactive marketing?
The answer is surely “no.” But it's important to recognize
what the expert's role is. The expert is not there to predict the
winner—as was demonstrated, this is very hard to do. But experts
play the critical role of using their experience and expertise to
narrow down choices to get to a bunch of finalists that, based on
experience, have a high probability of winning.
The reason our game was so tough was because all the values shown
were in fact "finalists," culled by experts, that had shown evidence
of success in previous experiments. For example, in the submit button
question, the copy choices included "Continue," "Submit," and "Click
Here to Submit." Guessing which of these three would perform best
is surely not obvious (in this case, the winner was "Continue").
However, if our expert dog had given us the choices of "Continue,"
"Arf Arf," and "Bow Wow Wow," guessing the winner would have been
a lot easier (at least I think it would... we probably should test
it to make sure).
This actually is very similar to the examples discussed earlier.
Take American Idol. While Simon Cowell, an alleged expert,
may have a tough time predicting which David will score better with
the public, he provides tremendous value in narrowing the original
field of thousands of performers to a select few.
So use the experts to narrow the field, but expect there to be
upsets. The only certain thing in life is that life is uncertain.
Likewise, the only people who can tell you definitively how to maximize
conversion rates are the people themselves. |
