Blackbeaks Blog….All things Analytics

An old friend of mine Sean D’Souza was in Chicago recently watching Tom Leung (Google Website optimizer team) do a presentation about the benefits of website optimizer. Tom thought that Sean had some Chutzpah when he offered to show him what he was doodling during the presentation. Sean however wasn’t doodling, he was actually putting one of his many talents to use to describe what he was seeing.

My favorite is this cartoon;

Too much testing can make things messy! Don’t try to juggle too many variables at once, do simple tests that are designed with a simple purpose in mind and then if it’s improved things, move onto the next test. If it hasn’t improved things then go back to the beginning.

When Sean showed me this article I just thought it was cool! It gave me a chuckle and shows simplistically what people should be thinking about when doing A/B testing.

Looking at the rest of Sean’s drawings I had slightly different thoughts to Tom, though I’m sure Tom was relating to his own seminar speech and putting the correlating discussions into the context of Sean’s cartoons. However I thought I’d share what i thought when I saw Sean’s work.

Be patient. Long term positive trends are what you’re looking for.

This is a great one. The slow and careful tester will often get better results than the tester who tries to test everything at once. It’s true. You have to be extremely careful not to “overdo it” when doing multivariate testing. Take it easy. Do a tortoise impression.

Simple tests with optimizer such as A/B testing can be very effective. A/B testing is like comparing one thing to another and seeing which is best. A bit like comparing to girlfriends!

This is another cool one. Do bold changes. I would advise this on sites which are really awful at converting their traffic and need to do something radical. I wouldn’t advise sweeping changes on websites generally because you can never tell what has worked if you do too much at once. However being bold doesn’t mean sweeping changes in my opinion, it means doing things differently and trying to test an idea or a concept with something that stands out from your control (your original site).

And finally…..

I completely agree with Tom on this one and have often been in this position. On the one hand I see a massive bounce rate and think “that’s a huge opportunity for improvement” and on the other I think “God that is awful! how can they be so bad at the basics!”

Combining web analytics data with testing is a great way to decide where to make your changes and I’d start by looking at the highest bounce rate or the page with the highest number of exits from your site.

Oh and just in case I forgot to mention, just like Google Analytics, Google Optimizer is free. So this won’t cost you the earth.