One complaint I hear from clients and students all the time is that their website’s traffic reports cannot possibly be right. Or, on the other hand, I watch them counting ‘visitors’ as if the numbers represent actual people. Neither is true. Or completely wrong, either.
(Digression: Winston Churchill once said that anytime you put two economists in a room you have two opinions. Unless one of them is (was, obviously) Keynes, and then you have 3. Opinions about online traffic stats are just about as varied as this — you need to get used to it.)
What got me started on this is a quote from Guy Kawasaki in an interview with Wordtracker’s Mark Nunney:
I learned that nobody really knows what their stats are because server logs, Google Analytics, and every service that purports to know “the truth” are all based on black magic.
Wow. Should we just give up? For years we’ve been talking about how online marketing is different because we can measure traffic all the way to leads and sales, picking the paths that are most productive. Now it’s down to ‘black magic?’
Not really, but there is certainly an element of truth in it. I am constantly dismayed to find Google’s traffic estimates or advertiser competition estimates wildly out of whack with what I can observe in my own traffic data or search results. Or, try a local search and 2 times out of three you get the 10-box (local results mapped with brief links adjacent) — but the 3rd time you don’t. Why does Google Analytics give me different information about my Pay Per Click account for a given period than AdWords itself does when the PPC account is directly linked via automated tagging?
These inconsistencies are not just ‘accidents’ or random noise introduced by the probabilistic functions of some overly-smart search engine’s algorithm, even though some really smart engineers might be able to ‘explain’ the anomalies. They are obtrusive evidence that we might not want to trust the search results, and maybe this is where some people get, just deciding that the search engines are a game to be played.
Well, I am a bit determinedly naive about some things, and this is one. The SEs and our analytics tools do measure things probabilistically, and there is error in the results. Plus, the measures depend on how well the hundreds of thousands (of millions) of hyperlinks are set up to track, and lots of them are not set up very well. And they depend on whether the signals arrive via the measurement tools to begin with, and sometimes they don’t.
Then, there’s the human variation, thank goodness. Humans have their own ideas about things like ‘cookies’ (good, OK, evil, dangerous, necessary, helpful, threatening) and they can delete those cookies whenever they want. Well, then the Google Analytics data is impaired for those visitors. Research is difficult on this subject, for obvious reasons, but people who have tried to do it guesstimate that maybe 30% of surfers delete cookies. That’s a lot of data. And I bet it varies in unknown ways by market segment.
But if we use our data to estimate magnitudes, compare large volumes, watch trends over time, or identify specific tactics that work over a long period of time with adequate amounts of traffic data, we will learn real, useful information about our marketing campaign. With enough time and traffic we WILL find out which keywords lead to conversions and which don’t.
Traffic stats are imperfect. But I don’t think anyone wants to trade them in for a billboard on the freeway.
