“I run a lot of home page tests, and they always come back 50-50!”
I heard this question from the audience during a great panel a few weeks back with Timberland, Red Envelope and The Discovery Channel talking about their experience with testing.
The questioner was not alone. His plaintive tone was met with murmurs of assent from amongst the crowd.
The fact is that most companies make their first foray into live testing by showing two versions of a home page (often one for a week, then another). And sadly, many of these folks find that both versions perform equally well (or equally poorly).
take heart: testing really does work, and we are here to help. Here are four ways in which home page testing can be tricky, and how you can overcome these challenges.
Challenge #1. Too many changes at the same time
When you change everything in version B of a home page test, you learn little. I won’t name names, but I know of companies who have shown two versions of a home page where layout, promotions, copy, and even products were totally different in the different versions.
Any one of these changes, taken alone, could have helped or hurt conversion in the new version. Taken together, their effects work against each other: the product would have increased conversion 12%, but the copy was bad enough to erase that advantage.
To visualize this, try testing the effectiveness of tow trucks by attaching one to the front of your car and one to the back at the same time. Just because the car isn’t moving doesn’t mean the trucks aren’t powerful.
Solution: Try multivariate approaches that can isolate the effect of different elements. Perhaps part of the “old” page with part of the new is the best recipe.
Challenge #2. It's a long way from home page to conversion
The distance from the entry point of your site to the point at which a prospect actually reaches the conversion point (a form page to fill out, the shopping cart, etc.) can be immense.
Think about it: a prospect goes online to look for new houses in his favorite neighborhood. An ad takes him to a company's home page, and from there he navigates about, looking for houses, checking out interest rates, comparing schools. By the time he finally decides to sign up for email blasts that alert him when new houses come on the market, the home page is long forgotten.
Just as a good pick-up line may initiate a conversation but won't get you married, a good home page may bring a prospect deeper into your site but is unlikely to get him to convert.
Solution: Change your measurement of success on your home page test. Forget about actual conversions for now, and instead see if you can improve penetration into the site. Did the test increase the number of prospects who visited certain pages? Does it seem to be motivating people to take the next step down the funnel?
Challenge #3. Your home page serves too many segments
Practically everybody who comes to your site, no matter where they originate, at some point passes through the home page. There's an enormous fragmentation of interest. For example, you've got
men vs. women
prospects who clicked an ad vs. prospects who typed the URL vs. prospects who came from an affiliate
prospects who came directly to the home page vs. prospects who landed on an interior page first
browsers vs. committed buyers vs. ambivalent users
The new elements you are testing may actually attract people, but also may alienate others at the same time.
Solution: First, if you want to test how you can motivate all of these prospects across the board, you should stick with big, significant changes such as a dramatic ease of navigation or a promotion that is of general interest to all.
Your other opportunity is to show different content to different segments visiting the home page. We use Offermatica targeting to focus the test on only new visitors, or only visitors from certain campaigns.
Run different tests to each of those segments. A little focus makes aiming much easier. The net result is that you'll get a cleaner answer on the test if the population who sees the test is more consistent.
Challenge #4. Lack of courage
The biggest challenge in home page testing, in the end, is just timidity. Good ideas are edited out of the process during meetings as being “bad ideas” before they can be tested. The result is a test that is really an A/A test -- there may be a little tweak here or there, but the two versions are really the same thing.
Solution: This is the hardest one of all. The inflexibility of most corporate web sites has led to habits where people assume that ideas have to be killed early.
With testing, you can let your customers kill bad ideas and reward good ones. Open up the spigot a little bit and make changes that matter. After all, are you in business to win an argument at a conference table or to build customer loyalty and sales?
Don't give up on the home page test simply because your results were less than perfect. Use what we’ve learned, try again, and you'll begin to see the needle move.