Blackbeaks Blog….All things Analytics

In my experience, the same thing applies to working with designers creating digital products. Just like in any design practice (architecture etc) there are a lot of best practices, which, in combination with designer’s experience, can create a very good initial user experience.

However, even if you create version 1.0 based on a gut feeling, you still should be able to set clear targets. Not setting targets because “it’s too complicated” is a poor excuse – and I firmly believe these targets need to be set very early on, also as a guidance for the designer as to where the emphasis should be put.

Thanks for the thought-provoking post, Steve!

@Leevi;
Once upon a time there were two designers and they showed me a superb concept painstakingly designed to improve sales of a particular service business they were working on. I watched and I nodded and watched and nodded then I asked what I thought was a perfectly feasible question….

It all looks great but how do you know it will work better than the current site?

This was 2006 and the looks I got would have literally flayed the skin from my back if I hadn’t already got a thick skin from previous discussions with designers!

My point then is the same as I was trying to make here. I fully believed the guys sitting in front of me were talented professionals with rich design experience, I could tell by what they were proposing that it was going to be far better than the previous version and I wasn’t trying to insult them. I was simply suggesting that they benchmark and prove their efforts were better than before. So by taking some “before” measurements they could prove that their gut feeling and their experiences were right to their client.

Once these guys got over the fact I wasn’t trying to insult their professional integrity and showed them a couple of examples they fully bought into measuring because in most cases the reason they were hired in the first place was proved by the numbers. However on the occasions where they got something wrong they could move to “plan B” and for a design professional it’s always good to have a fall back.

I agree with you that targets as in all aspects of life are essential in my opinion. Like the Cheshire Cat from Alice in Wonderland said… “If you don’t know where you’re going, any road will get you there.”

The precise reason for having a target destination is knowing how to figure out which map to use.

Well put! Analyst types often treat the phrase “I believe” as heresy. But, I find myself using it all the time when talking about performance measurement of social media — if you insist up front in a valid, fully vetted ROI model for calculating the direct financial impact of your Twitter or Facebook investment…you probably won’t invest in either channel. But, if you look at all of the secondary research that shows that’s where consumers are heading, and then if you put together a logic model / value chain (“beliefs”) as to how engaging consumers there will ultimately help your business, then you can have a crystallized objectives for that investment, and you can measure whether or not you are achieving that objective. But…you may still have to rely on gut/experience/beliefs to say, “And those results are meaningful.”

Or…did I just extend your point about “gut” in a way that is actually heretical?

I’m gonna print that quote from Alice in A3 size and hang it next to my desk

On the ability to support gut feeling with numbers: too often the things that slip through the cracks in the measurement specification are the ones designers come and ask tough questions about. So even though it might seem like a waste of time in the middle of site launch frenzy to tag every possible interaction on the site, I firmly believe it pays off. A recent example is the image gallery section of Yle’s microsite for Finnish independence day celebrations, at http://yle.fi/linnanjuhlat/. The tech team did excellent job in tracking every aspect of the gallery as events (images loaded, navigation patterns, etc). The team behind the site concept knew from experience that good quality images from the celebrations are a major motivation for the public to go online already during the celebrations, and this year the images were put online faster than ever before with help of streamlined uploading process. Afterwards, with the help of very detailed data, we could e.g. segment the gallery traffic, and look at it hour-by-hour through the night and the following day.

It was easy to prove that the gut feeling of the team was correct–and what’s even better, we now have data in store for next year’s site on what works, and what can be done better!

A couple of days ago Anil Batra posted “HiPPO: Wrong Advice That Will Get You In Trouble” and now you post this great call to reason.

Could 2011 finally be the year of “enlightment”? Managing solely by guts is as wrong and crazy as blindly following what the numbers say. Analytics role is to help managers make better decisions, not overrule them. When brought to a level of blind faith and religious status, analytics, like anything else, becomes dangerous…

Thanks for a great post!
Stéphane

@Tim;
No I don’t think that’s heretical though it’s typical that as analysts we all agree that working from the gut and intuition is a good thing and then come up with new and interesting ways to measure them!

@Leevi;
Haha! Not only do we come up with new methods to measure our gut feelings and intuition but also make damn sure we don’t miss out on proving whether we were right or wrong by ensuring tagging is perfect prior to testing out our theories.

Are we Nerds or what?

Great post! Very true that we have a big problem on our hands. Once we get data, we try to make it fit everything and everything fit it. But that’s not how the world works. At best, it’s a support role (and a very good one), but it’s not a guidebook for business, else everyone would do the same thing and have 100% conversion rates.

I had the same feeling in this post (http://www.atlantaanalytics.com/web-analytics-in-business/becoming-a-data-driven-business/), would love to hear your thoughts on it.

@Stephane;

Thanks! I agree with Anil Batra too – check out the post that inspired him to write (and also the post that inspired me which is by far the funniest!). I hope you’re right and 2011 is the year of enlightenment.

@Evan
Yes if it were as easy as looking at data we’d all be laughing on our way to the bank. I guess the trick is getting the balance right, knowing when to trust yourself and knowing when you really need data. I’ll take a look at your blog and comment there.

Hi Steve,

I agree that web analytics requires a mix of intuition and data. It’s the Art & Science of our industry. The art allows you to hypothesize and come up with creative ideas. The science determines if your ideas worked. We place measurement here (much like you convinced your designers), to substantiate the gut feel with hard data.

Nice Post!
John

Hi Steve,
I agree with you that gut feelings has a role to play in our community. Having said that, gut & intuition does not arise out of vacuum. The more experience one garners, the more (& better) their gut feelings become in terms of reality & practicality.

From this point of view, analytics & data is important (especially for a novice). It allows us to understand the bad judgments we made so that we “block” the next intuition in that direction and also allows us to learn from what worked which then is fed into our “gut’ .

I believe that gut feelings, intuition, creativity is needed to come up with scenarios and hypothesis but I would not bet a huge investment on it without some analytical due diligence. In the end, how one combines gut feelings with analytics would depend on the context, audience, time available to make a decision, and the risk of a bad decision.

Nice post.

Regards,
Ned

@John,

Thanks for the comments. I like the way you describe it as the art and science of our industry. A good analogy.

@Ned,

I agree knowledge doesn’t rise from a vacuum and that analytics can generally help teach folks understand what is working and what isn’t. No question.

But I think knowledge comes from a wide range of sources and experiences outside of the numbers and sometimes you just have to dive in and learn. I am not advocating making massive investments I’m talking about testing what you think.

@all – the reason I wrote this post…

I met an 18 year old a few weeks ago with some outlandish ideas for a web service and I told him…. “I don’t know if that would work. But what the hell do I know? Try it. Analyze the results, use GA and a testing tool and try it out.”

Ned’s right in a way that what I asked him to do was test the hypothesis before making a massive investment and I would always advise it if it could be done.

But I don’t think for instance FaceBook started that way, or Google, or Apple, or Microsoft. These guys just had guts, belief and eventually before the huge investments happened they got the proof that they were right. What they made was cool and transcended numbers. The numbers came afterwards.

Hi Steve,
Just to clarify – you made a fundamental observation in your comment to me which I totally agree with. ALL KNOWLEDGE DOES NOT COME FROM DATA – this is very true.

Viewing from this lens, yes – one should not be jumping into conclusions just based on analytics or paraphrasing it, analytics by itself is not a panacea that can provide answers to all the questions or issues one might face.

Cheers,
Ned

[...] [...]

Wrong? You are destined for failure if you don’t listen to your 6th sense because it almost always right. If it’s wrong it’s because you missed something you didn’t want to get it.

When it comes to being analytically correct, listen to your 6th sense when it says you don’t know nothing!

Mine is telling me you guy will love this site if you don’t already go there http://whichtestworks.com

See? I listened to my 6th sense just now when it told me “works” didn’t look right and sure enough here’s the correct one!!
http://whichtestwon.com