The Secret Multivariate Testing vendors don't want you to know

2 Comments

Aug

12

2010

by MrOptimization in Numbers

Bones - MrOptimization.com

Nobody uses the thing!

Over the past few months, MrOptimization has performed a quasi-scientific survey of online businesses to try and discover how they're utilizing Multivariate Testing and Targeting technologies.

We evaluated over 100 digital properties and were surprised to learn how little benefit most are getting from their investments in these potentially game-changing technologies.

How do you make money online?

The digital properties we evaluated represented most major revenue models online today.

Multivariate Testing Vendor Usage Survey #1 - MrOptimization.com

How long did it take before you ran your first test?

Various reasons were cited for the long delays between signing the contract with a Multivariate Testing vendor and actually launching the first test.

Multivariate Testing Vendor Usage Survey #2 - MrOptimization.com

How many tests do you run?

Once up and running, a huge majority are barely using the technologies.

Multivariate Testing Vendor Usage Survey #3 - MrOptimization.com

How are you using the multivariate testing vendor technology?

Despite the powerful Multivariate Testing and Targeting features available in most mid to enterprise-level testing platforms, a majority of the digital properties we evaluated are only using the most basic capabilities.

Multivariate Testing Vendor Usage Survey #3 - MrOptimization.com

What can be done?

What are the reasons behind these depressing results and what can be done to get real benefits from these investments?

Stay tuned!

2 comments

Tom
Interesting
Fri August 13, 2010   11:41:19
assume this is related to two limitations: resources and understanding of the technology. having recently signed a contract with optimize, we'll see if our mileage varies. but in researching software, we found the presentations and promises of potentially huge ROI gains were very appealing and i'm gonna go out on a limb here and say that most of the people in the green section on these charts were sold on the idea but have no idea how to use it.

i also have a hunch that those that do are realizing that testing and planning takes time and likely aren't willing to commit the necessary resources to back the platform.

would love to see what some of these companies site as pitfalls so i can avoid them as i begin the process myself... but assume those same people aren't likely the ones reading this blog :o
MrOpti
You nailed it, Tom
Wed August 18, 2010   07:05:40
Tom,

You're right, the #1 reason cited by respondents for the atrophying of their optimization program was some form of "resource constraint" or "demand from other projects."

In fact, they pretty much fell into the general categories we covered back in May: Top 9 Signs your Multivariate Testing and Website Optimization Efforts are doomed

We'd love to hear how things go with your optimization program once it's been running for a while. Sounds like you're approaching it the right way from the start.

Thanks for getting in touch.

-MrOpti

Add your comment