Blackbeaks Blog….All things Analytics

Steve,

Don’t you ever sleep?

I was pretty surprised to see Avinash take a complete about-face on the subject, weren’t you? On one hand “engagement is an excuse” and on the other, well …

“Following a number of leading Web Analysts I also believe that a customer’s degree of engagement is better calculated as a synthetic metric composed of several basic metrics, rather than as a one-metric solution e.g. measuring customer engagement by means of ‘duration of visit’ only.”

Nice to see he’s come around, huh?

I look forward to reading more (hint, hint) about REAN and I talk about your model quite often lately. Nice work!

Hi Eric,

Sleep is overrated!

Yes as you later pointed out (on Occams razor) it wasn’t actually Avinash’s opinions on the matter but rather Theo’s. I do however know Avinash has seen and appreciates REAN and in that context can’t fail to see the logic behind the engagement formula you created.

By publishing Theo’s thoughts Avinash is showing that he is open to the views of others (you and I included) in the industry around this controverisal subject.

My view is quite solid. I use the REAN dimensions to cross reference KPIs and segments and because of that simply look at the whole debate as a problem with semantics, what one man calls engagement another calls getting married ;).

I value the discussion because it allows me to introduce the REAN dimensional concept and try to explain that by using something like this (it doesn’t have to be REAN - HP call it something totally different) you can better define what you’re talking about.

Thanks for the comments
Steve.

Steve,

can you please explain to me, whether here or in Avinash’s blog, a little bit more about what you mean by ‘I use the REAN dimensions to cross reference KPIs and segments and because of that simply look at the whole debate as a problem with semantics, what one man calls engagement another calls getting married’. If you dont have the time to do so, is there some document you can point me to?

Thank you
Theo

Hi Theo,

Sure I’ll do so here.

REAN dimensionally covers the whole lifecycle of customer interaction. In it’s simplest form for illustrative purposes it might go something like this;

1) Unknown visitor types a keyword into Google and visits my website as a result. (Reach)

2) I track his/her behaviour on my website simply monitoring click depth and time spent (Engage)

3) Visitor takes an action and registers for my newsletter as jon@doe.com (Activate)

4) I contact the visitor again via my newsletter, Jon opens it and re-visits my website whereupon I track Jons behavior (Nurture).

ALL measurements (regardless of the type of measurement index you produce) will fall into 1 or more of these dimensions.

Because I am using the REAN dimensions I have specifically pre-defined what Engagement is. It’s measuring the interaction of unknown visitors before they have converted. Only later when the activation has taken place is when it’s about nurturing prospects and customers which is what Jim Novo (for instance) is so good at.

What I meant when I said I cross reference KPIs is that I might have a KPI called Engagement Index which is essentially a segment based on behavior I have pre-defined as useful to look at. A level of click depth and time spent for instance on sites with millions of visitors a day is one such segment which can be used to seperate potentially useful traffic from not so useful traffic.

By then cross referencing Reach sources with Engagement I can determine which reach sources are more likely to engage. But you don’t stop there. By then also cross referencing which of the engaged visits activate and then become nurtured (later) I can also determine from millions of visitors where and who my best prospects and customers are.

REAN has two real purposes 1) to allow you to plan on and offline marketing activities around a defined concept and 2) to help plan online and offline measurement frameworks together.

As i’m working within pre-defined dimensions I don’t have the semantic problems associated with “defining engagement”.

Does this explain it?
Br
Steve

Thank you for taking the time to answer in such a comprehensive way. Its seems very interesting and i would very much like to learn more about it - ill read the WAA document you refer to. Any other links would be greatly appreciated.

I do not understand how you view E.Peterson’s work in relation to REAN. That is, though i understand that you distinguish what you refer to as ‘engage’ metrics from ‘activate’ metrics, and find it useful for planning purposes, i do not understand where you stand in relation to aggregating them to produce a synthetic metric like E.Peterson’s. Are against it? Is there a place for it in the REAN framework?

Finally i dont think i am engaged in semantic issues. I simply try to understand exactly what web analytics can tell us about the behavioural data it is concerned with, in this case with respect to a user\visitors involvement with his object of use/visit.

Thanks again
Theo

Hi Theo,

If you look at the component parts Eric’s formula;
Click-Depth Index (Ci) Recency Index (Ri) Duration Index (Di) Feedback Index (Fi) Interaction Index (Ii) Loyalty Index (Li) Subscription Index (Si)

Now lets simply re-arrange them to make it work in the context of the REAN model;

R - Brand Index (Bi)
E - Click-Depth Index (Ci); Duration Index (Di) Interaction Index (Ii)
A - Subscription Index (Si); Feedback Index (Fi)
N - Recency Index (Ri); Loyalty Index (Li)

When you look at the component parts of Erics formula it shows clearly that they all fall into one part of the model. This is why I always said from the beginning that Eric’s formula is quite a comprehensive one, it’s the first single calculation I’ve seen anyone using that takes every dimension into account. In my experience (11 years in online marketing 9 of those using web analytics tools) I’ve never had to use this complex a KPI yet. I tend to find myself using primarily REA (campaign type) metrics, or nurture metrics cross referenced against the other 3 dimensions.

I didn’t say you were engaging in a semantic issue, nor did I mean to imply it. I am just saying that by using REAN in this way you’re not bogged down with definitions as it’s all pretty clear cut. Avinash made a bold statement though when he said ‘engagement was an excuse’ which in the end after our little online debate about it I put down to semantics.

Web analytics can tell you about the behavior throughout the lifecycle as I’ve tried showing above, however it is always only going to tell you so much, even if you’re Jim Novo! :).

I completely concur with everyone who says qualitative data (surveys, onsite searches, interviews, heuristic & usability studies) can help you to understand why the behavior has happened and is very insightful.

Looking back I don’t think I’ve disagreed with anyone (even Avinash when I see his line of thought), just offered a different perspective based on the REAN approach.

Hope this helps.
Br
Steve

This definately helped. I can now see very clearly the relation between REAN and Eric’s formula and think it is very very interesting, so interesting in fact that i am thinking of writing a blog post about it if you dont mind.

Of course i am now led to ponder why you have not needed to use this as a single complex metric, or for that matter what the value of using a single synthetic metric is, which is what i am most interested in (especially after this discussion). In particularly i am thinking about whether there needs to be a differential weighting of different components of the engagement metric e.g. if visit metrics (recency/frequency/depth) should be considered of lowered engagement value than Activation metrics (subscription, feedback).

I also see very well why a definition of CE is not needed within a REAN framework. I now think that my attempt at a definition was simply intended as a means of clarifying what aspect of ‘engagement’ behaviour web analytics can capture. Other than that and given that it changes across companies i do not think it serves any other purpose really than a kind of segmentation such as the one i have performed here http://agoraplace.wordpress.com/2008/01/04/defining-customer-engagement/

Thank you very much for taking the time to explain all this to me,

Best regards
Theo

Hi Theo,

I’ve never needed to use as complex a KPI before because most of my clients aren’t at this level yet. Most understand basic levels of KPIs but don’t have all the required technology or expertise to really follow that kind of work through. Think about combining CRM databases with web analytics tools and scoring methods and you quickly start looking at more difficult scenarios to explain to folk that aren’t ready for it yet.

That’s not to say I won’t use it. I have one client who Eric’s approach is perfect for and we’ve discussed a way to practically implement it because a qualified lead to the client is worth millions.

So Blog away!
Br
Steve

Preaching about KPIs to my customers I have used more general one, marketing’s AIDA(S) formula. Someone even recalled that one! And sometimes, rather than talking about KPIs, I simply talk about important things and goals of the website.

Personally, I think REAN is overly complex, it seems what you are doing with R-E-A is simply creating classification buckets for different types of value. If that helps people understand the problem, that’s fine.

A challenge is because REAN is so web-specific, it will not translate well across lots of channels and business models. At some point, you have to bring all of this together in a logical way so it can be analyzed, and by creating additional web-specific value buckets barriers to this are created.

In the long run, no matter what business model or channel you are dealing with, visitors / customers have 2 types of value: realized (REA), and Potential (N). I would consider REAN a web-specific subset of this larger behavioral model that already is pervasive in the BI world.

If having a web-specific version of this model stops the squabbling about Engagement - that is, the problem is really defining engagement, not understanding the dynamics of acting on what it tells you - than so be it.

But again, personally, I think people are simply making this idea too complex. Nothing could be simpler than asking two questions:

1. What is the value created - however you want to define it or measure it, that doesn’t matter one bit. Leaving this question of how value is defined completely erases all reason for squabbling, IMHO.

2. What is the likelihood of this value recurring?

Jim: Is not the attempt to better understand what web analytics can tell us about engagement an attempt to better understand the likelihood of this value recurring?

<b>Petri</b>,

AIDA is what REAN is based on. However I started using REAN because we couldn’t define how to measure interest or desire without excellent qualitative data.

<b>Jim</b>,

You’ve hit the nail on the head. REAN was not even designed for the web I simply built on the framework for that purpose. It was originally designed to plan multiple online and offline campaign activities and then plan/show how to measure them.

I’m currently working with one REAN model across a campaign which takes the web, radio, TV, PR, Email DMs and Mobile advertising. It also has number of engagement points (A mobile site, a website & interactive event screens) about 50 activations and a number of Nurture metrics - such as email forwards, mobile MMS % lift (the main campaign business goal) in recipients 3 months after the campaign for instance.

As I commented earlier;
REAN has two real purposes 1) to allow you to plan on and offline marketing activities around a defined concept and 2) to help plan online and offline measurement frameworks together.

As for your questions, I agree with them both. Yet we still see this discussion on blogs all over the world.

@Theo, just because an event happens, doesn’t mean it will happen again, and that is the problem with all the focus on “activity” as opposed to “likelihood of activity”. If you are only doing #1 above, you’re not really doing anything new or different, you’re just taking the same metrics and making them more complex. That may have value, but it’s more of the same - measuring activity.

When you add #2 above, now you are talking about the relationship, something that has value in the future, something new that is worth measuring because it can dramatically increase ROI.

For example, this likelihood absolutely plugs directly into all the talk about “relevance” we hear about today. Customer gets screwed in a customer service event. Customer thinks, “Stop sending me e-mail, I don’t want it anymore, you aren’t relevant to me anymore”. Don’t you want to predict this dis-engagement event, rather than have the customer complain about it or maybe post bad stuff about you on some blog? You can’t do that by measuring activity, you have to measure the *lack* of activity or make some other kind of prediction.

@Captain, First, let me make it clear I think REAN is a nice piece of work.

But, as Petri alluded to, REAN is really a web-based manifestation of a sales process model, of which there are many, including AIDAS. REAN takes these models and turns them into web-specific actions and data points which many other media lack. Not that the customer doesn’t go through the process, it’s just next to impossible to *measure* offline.

All of these models speak to the higher issue of the Customer LifeCycle, which futher begs the Relationship Marketing model that started all of this talk about Engagement by Regis McKenna in 1991.

A question is this: If REAN is a planning model, then it is a structure you can follow to map and act on the Customer LifeCycle, which is great and useful. But if REAN is a “planning model”, is it measuring anything? Does it answer the question of “how do I measure Engagement”?

Jim

Thanks for the comments. Your predictive BI thinking is relevant to the idea of REAN also. This is because it depends on the KPIs you develop. If you want KPIs that measure customer dis-engagement then the N part of REAN would definitely be the part of the lifecycle that you discuss.

REAN is a planning model for both the lifecycle and and the metrics but it is definitely not measuring enything at all. It can however help answer the question “how do I measure engagement” by allowing your planning process to be clear and defined.

I could have activity based metrics in REA and likelyhood of activity based metrics in N and actions taken to build relationships as part of your KPI flag parameters (if ‘x’ happens, do ‘x’). A stage further would be looking at the REA activities measured which increase the likelihood the further N actions will be taken. So I believe it can help answer the question.

Yes it’s a sales process model, but that’s what it’s all about at the end of the day, sales. Regardless of your expertise (be it predictave modelling, CRM or demand generation) the end goal of all business models is sales, at least eventually.

What I’ve found REAN useful for is planning, but none of my customers ever get into a conversation with me about “how do I measure if they’re engaged?” because we define these four dimensions in the KPI planning stages.

This is what I am driving at.

Jim: Its not so much my lazyness to send an email but i think this question would interest others as well: Is there a place where you have commented on the value/usefulness of single synthetic metrics such as E. Peterson’s?

@Captain - so I think we are in agreement then, that REAN is a (very good) framework for the implementation of the macro ideas surrounding Engagement metrics - the Customer LifeCycle, Relationship Marketing - but in itself is not a measure of Engagement. As you stated, there are still underlying KPI’s that need to be constructed, which is where the whole wrestling match starts - when you get down to KPI’s.

I did not detect any resistance to the idea that if we are going to measure Engagement, that implies a relationship, which implies “value over time”, which drives towards the idea that “likelihood to continue” is what differentiates measuring Engagement from measuring Activity.

So Activity defines Value:

You can have Best Customers or Endorsers or whatever you want to call them for the environment you are in - your highest value participants. At the other end of Activity / Value, you can have Low Value Customers or Detractors or whatever you want to call them. Their Activity defines their Value.

And then Likelihood to Continue defines Engagement:

You have various levels of Engagement, a continuum from Highly Engaged to Formerly Engaged, which is a prediction of Value in the Future. So you can have Endorsers who are Formerly Engaged, you can have Low Value customers who are Highly Engaged - and buckets in between. To optimize the system, you would use a different Marketing strategy for each segment, which is the whole idea behind Relationship Marketing. You could then step up to REAN to implement Relationship Marketing based on the Customer LifeCycle.

In the implemetation of REAN, I take it you use some kind of multi-point analysis to paint a more complete picture of the LifeCycle - they are Enagaged here, but not here, etc. The “not here” is essentially a prediction of Future Value through that touchpoint. So you get the notion of a customer can be Engaged overall, but dis-Engaged with certain touchpoints, meaning there are LifeCycle changes going on relative to that touchpoint and the strategy for that customer should change, e.g. they have converted from an offline relationship to an online one, and offline is not relevant to them anymore, so stop sending snail mail.

I’m hoping painting the picture as a whole system, rather than just an “Engagement Metric”, helps people understand the basis for why the idea of Engagement is important and how it flows through the Relationship Marketing framework. Said another way, the core concept in Relationship Marketing is *relevance*, and relevance is measured by Engagement.

@Theo - I commented back when Eric published his idea that I think down the road, and especially when you are talking about multi-plaform / channel / whatever, that combining the two vectors of Realized Value and Potential Value could be problematic because they are two different things; one is tangible fact, the other a prediction.

That said, my viewpoint on this issue is more from a “Standards” or educational perspective, because I think if you really understand the inner workings of what you are trying to measure and accomplish and can “see it in your mind” - which Eric clearly does - then you can go ahead and create synthetic shorthands. A lot of times this is done to simplify presentations to Senior exces, for example.

But I wouldn’t teach this synthetic idea as the base case, because the level of abstract thinking required to interpret it correctly is (in my opinion) too high. I would keep the two value vectors separated for clarity - this stuff is already complex enough thank you - and leave synthetics for the Advanced class!

By the way, I liked your thinking and work on Kind and Degree, and agree that if you are looking at these kinds of parameters, you’re not going to find them in your web analytics tool. That does get problematic from an implementation perspective…:0

Perhaps you and Joseph C. can refine this idea a bit more. I can see a discussion around these kind of parameters taking place once you have identified specific dis-engagement behaviors and wanting to know more about “why” - surveys, focus groups, etc. Your framework could provide a “map” for those discussions much as REAN does for the LifeCycle. But you have to be able to tie these insights directly to specific behaviors you can measure in a tool or you run into the actionability problem - what is trigger, how do you act?

So I would see your structure used not to trigger action, but to answer the question of “When we act, what should we do?” It’s a complimentary tool set that would sit inside a framework like REAN or AIDAS.

Nice work on thinking that through.

Thank you Jim. It gives me great confidence that you thought my post in Avinash’s blog was interesting.

To his credit, Steve asked me several times to write a few lines on how i first came up with this REAN framework and what its intent was - which i never did. Well, let me try to correct this now.

I came up with this loose framework while i was working at Satama. The basic point was to get a large client of mine grasp the very simple idea indeed that there’s no point at all in investing shit loads of money in creating cool websites if no one was looking at building traffic for these sites in the first place, if no one was then defining what to do with these hard-won contacts, how to leverage them as part of an entire branding, marketing or sales effort, now and in the future, online, but also offline… (because, yes, as much as it hurts to recognize when you’re working in a web agency, there IS life outside the web. and guess what: most of it still happens THERE!).

In other words, the two messages i wanted to deliver were:
- break the silo walls: it’s not SEM + web site + CRM + …. it’s about creating EXPERIENCES that flow from the very first moment a contact is established down the lifecycle of that contact. and this, especially in large companies, often implies difficult organizational alignment efforts and clear leadership
- define what your objectives, business goals are, think hard, before you do anything and invest any dollar. Then only it becomes possible to design effective & efficient marketing ARCHITECTURES. creatives are cool and certainly needed, but without a good, solid architecture, they’re pointless.

Initially, this framework was thus more of an educational material than anything else. However, we instantly started to use it as a tool to structure our work and ground our metrics. This was my contribution.

The metrics side of it was developed first as a collective effort with Steve and a few other former colleagues, notably Leevi Kokko (now working for Ego Beta), and then taken to a new dimension by Steve alone.

But hey! it’s really cool to see this idea lives and develops.

In my upcoming book (which is now about 1/3rd done) I have credited Xavier as the person who invented the REAN framework when working at Satama (as I mentioned in my initial post above) back in 2006. I’m delighted to see he contributed here - I now have a solid quote for the book! :).

Credit where it’s due, Xavier’s initial vision has been a foundational framework for a lot of the things done at Satama. I use REAN for planning purposes in campaigns, for KPI workshops, for Hub and Spoke process modelling an even for quality checking processes around tagging!

It’s become a solid tactical method in the larger hub and spoke process we build for our clients.

As soon as I saw the framework it I knew it had great potential, particularly the way that Xavier and Leevi had used it to start developing metrics which meant the clients instantly understood why we proposed measuring what we measured without confusing anyone.

It was one of those things that made my job easier which is why I have put a lot of effort into developing it further and have now started (after 18 months of field work) sharing it with a wider audience.

@Steve, nice to hear a status report on the book (I’m sure the others are waiting for it as much as I am)!

Having seen you present the KONE case at eMetrics Stockholm last year, I was impressed by how you had improved the initial REAN model and utilized it very cleverly in the analytics process.

I remember the first realization very well, and from the very first project it was created for it really turned out to be an invaluable tool in customer communications and planning in many ways. I’m personally using a variant of the model with practically all of my clients now.