Blackbeaks Blog….All things Analytics

Have you ever asked your self whether the paid traffic is the one which is getting you most of the sales or whether it was your organic traffic that stole the ROI from your PPC campaigns? It’s ok, that sentence didn’t make sence to me either, but I’ll continue and let us see if the picture gets more clear in the end.

I was talking about the extra links couple days ago and wondering about their affect on conversion. That made me think of something that the great Jim Novo ones said in the forum of Web Analytics Demystified. There was a thread about Page views / Conversions / ROI from the perspective of the marketer. It was in that thread where he unveiled something really genious , which I have noticed to be pretty much correct in few of our client cases as well. Now, you are thinking: “Black Jack Vane, stop mumbling.. Get to the point man!”

I have decided to cite him directly from that thread, and hope that you don’t mind me doing this Jim. His exac citation goes:

“and the question if you are optimizing *profits* as opposed to
sales or “exposure” is where is breakeven on the PPC? This of course
depends on the margin of the business and the cost of the click, but there are several other dynamics in play, including the following:

1. The tendency of a 1st ranking PPC to deliver sub-optimal ROI due to “sport-clicking” by causal / newbie “surfers”

2. The tendency of lower-ranking PPC to deliver higher ROI due to bid gaps and (often) dramatically lower costs

3. The tendency of “deep searchers” – people who click on lower ranking organic and PPC links – to be further into the research cycle / more likely to convert to final objective

So, for example, in one 2003 test, a page ranked organically as #2 for a certain high-volume phrase. This same page content was used to create a landing page for a PPC campaign using the same search phrase. The test was conducted in March, neutral seasonality for this business.

With a #1 PPC ranking, the PPC campaign generated 11% in *incremental* sales with a 60-day latency tail but had a negative 12% ROI on margin minus overhead (note specific use of the word incremental, which I will explain further below). When this same listing was dropped down to “deep shoppers”
at PPC rank #4 (in this case, 1st ad at bottom of Yahoo page), it generated 4% incremental sales with a 1786% ROI on margin minus overhead for the same
60-day latency tail. That’s almost 18 : 1 payout. Without the tail
(first conversion only), it was 623% (almost 7 :1).

In addition, the “deep shopper” segment on average had a 70% repeat purchase rate as opposed to 58% for the #1 PPC position. So even the “tail of the tail” was better at position 4. This was on the higest volume search phrase for the site, so it made a huge impact on overall profitability.

Remember, the landing pages for both the high ranking Organic link and the
#1 ranking PPC link were exactly the same – layout, copy, all of it.

Now, the reason I specifically used the word incremental is we had control, which demonstrates a real dark side of PPC with Top 3 ranking Organic.

When the #1 ranking PPC ran with the #2 Organic link, the sales *volume* coming from the PPC link ran about 43% versus 57% for the organic link.
This ties pretty closely with some recent studies on click behavior (on average, 60% click organic, 40% click paid).

But dig what this really means: if incremental sales are 11% versus control (no PPC) and sales volume with PPC is 43% PPC, that means nearly 77% of PPC sales were **stolen from the organic side** – they would have happened anyway without the PPC link.

Factor this “media cannibalization” into ROI, and now we’re down around negative 48% ROI for the test #1 ranking PPC with organic at #2. For every
$1 we spend we lose 48 cents – 12 cents in tangible ROI, and 36 cents in “Opportunity ROI” – ROI we won’t get because we wasted the click budget by not using it to buy “real” incremental clicks.

Hey, just increase the budget, we’ll make it up on volume!

This is precisely the situation I was alluding to in the first post on this thread.

The visitor clicked PPC, landed, went Back, found Organic, converted. With all this occuring in the same session, it’s highly likely that paid click is a pure subsidy cost – conversion would have happened anyway. To peg this click as “source” gives credit where credit is (probably) not due, and leads to eroding margins through increased PPC subsidy costs.

So back to the original question, can you think of a solid way this incrementality could be tested?

“Solid” way? Well, I guess that would depend on the technology that was available and the sales volume you are talking about. Not sure you can truly “A / B / C” it without some significant bid management / search engine API technology. I’m not sure the engines would give up that kind of control – though I bet THEY have tested it at some level.

The above test was a 3 week manual “alternating days” test on Overture for a store with 500K – $1 million annual sales and average order size about $72, so there wasn’t a lot of room for high tech tools.

If you run A on Monday, B on Tuesday, control / C on Wednesday, then start over with A on Thursday and continue this rotation, the following Monday B will run and on Tuesday C will run etc., so by the end of 3 weeks you will have A, B, and Control data normalized by day – both campaigns and control ran on every day of the week. Not a statistically pure methodology, but not horrible, for sure – and cheap! As Robin said, “I’d rather do it unscientifically (e.g. pull the ads for a time period that is longer than the average latency and measure) and swallow the error rather than just “attribute” the conversion.”

If the result spread is significant enough, as it was on this test, I’ll give up points in accuracy to get closer to the “directional truth”. Each of the top 30 search phrases *where there was a top 3 organic ranking for the phrase* was optimized in this way with the results very directionally consistent across all phrases. It was almost always more profitable to have a lower than #1 paid ranking when a top 3 organic ranking was present.
Below the Top 30 phrases, some of the lower volume phrases produced inconsistent results which was probably a result of test methodology error / lack of frequency.

While it may not be “practical” for large scale retailers to test like this, you would think for certain high volume phrases it would be worth poking around it a bit given the potential for cost savings. Definitely not worth thinking about if sales is the focus, because (probably
unprofitable) sales will be lost without the #1 PPC listing.

Are there any analytics providers with a direct interface to the search engines capable of generating / controlling / measuring this kind of testing on a large scale? Any analytics providers that interface with complex bid management systems like Kevin Lee’s didit.com ? I don’t know the answer.

Of course, changes in the way paid listings are displayed (often related to how many bidders there are) can change the outcome of this test. The results on Google were also directionally consistent though less dramatic.
I assume this is because of the different PPC display approach versus Yahoo, but perhaps also due to some of the “alchemy” surrounding Google PPC rankings, which are more difficult to control on Google.

And for sure, there are reasons people buy PPC other than to drive profits, so being #1 may be worth it, but the true costs should be quantified.”
Ahar! Well said. And I agree.

- Black Jack Vane -