Greg Harmon on Web Traffic Numbers

My post earlier this week led to an interesting phone conversation with Greg Harmon of Belden Interactive. We discussed the apparent disconnect between publishers and audiences shown in the ITZ Belden presentation given at last month’s API confab, but mostly we talked about Web traffic numbers.

I’m math-challenged, so I was following Harmon into deeper waters than I’d like, but I think I followed along with him without getting too lost. His contention is that newspapers’ counts of unique visitors are drastically inflated by measurement errors. That’s bad news for publishers who like big numbers when discussing traffic and reach — but Harmon sees it as good news for advertisers and folks trying to figure out how to turn online newspapers into a business. First off, they need reliable data. Second, there’s a  silver lining to the measurement errors: Harmon thinks they substantially undercount visits per user, meaning those smaller audiences are much more engaged and valuable than papers have been led to believe.

Harmon says his efforts to get better traffic numbers date back five years or so, to work he did for Lee Enterprises. Lee was getting numbers indicating there were 400,000 monthly unique visitors to the Billings (Mont.) Gazette, which seemed odd given that the print Gazette’s circulation was around 50,000 and its total readership was probably 80,000. Was non-local readership really that high?

Harmon — with help from Alan Hubbard, a professor of biostatistics at Berkeley — took apart that data. The data about page views and visits seemed sound, so they analyzed the numbers based on what they knew about how many page views an average visit generates — a number that’s fairly standard across newspaper sites, though it changes on weekdays and weekends. Running those numbers yielded an estimate of average visitors per day that they thought was more accurate, and from which they could extrapolate. To discover where site visitors were coming from, they surveyed them through Web-site interstitials, which let them categorize readers by how often they visited and whether or not they were local. Harmon’s conclusions: The Gazette’s average monthly traffic was really around 40,000, and half of that came from Billings.

Harmon said the news brought cheers. Why? Because while 400,000 monthly unique visitors was impressive, it was useless for selling ads: It made the print salespeople’s 80,000 readers look puny, and stuck the Web salespeople with a number that advertisers doubted and that they couldn’t explain. The new numbers, he says, let the Gazette present a unified marketing message, and gave potential Web advertisers a realistic number to consider when thinking about their potential return on investment.

But why were Belden’s numbers so different? Harmon looked through data from some 60 sites Belden Interactive had worked with, and used surveys to focus on potential factors. Site visitors were asked how many computers they’d used to access a site in a given month. The average answer was two, but that wasn’t enough to account for the discrepancy.

Harmon’s suspicions fell on cookies. Were site visitors clearing them? He suspected so, even though newspaper executives and techies disagreed, arguing that people didn’t habitually clear cookies and most had never heard of them. (I was skeptical myself. And the big audience-measurement firms would say that cookies don’t affect their metrics.) Harmon says when questions about cookies were added to Belden’s surveys, about 50% of respondents had indeed never heard of them. But 25% of respondents said they cleared cookies weekly, and 10% said they did so daily.

Such small percentages can drive big errors, he explains. Say you have 1,000 readers who visit a paper’s Web site once per day. According to Harmon’s survey results, 500 of them have never heard of cookies, and so will be recorded as 500 monthly visitors. Another 150 have heard of cookies but don’t mess with them. They count as 150 monthly visitors. But 250 visitors clear their cookies weekly — and so they show up as 1,000 monthly visitors. Then there are 100 visitors who clear their cookies daily. They show up as 3,000 monthly visitors. Add them together, and 1,000 monthly visitors have become 4,650, even though two-thirds of the audience never messes with cookies.

Harmon says he sees evidence of this in the relationship between page views and unique visitors reported by papers of all sizes: Figures from paper after paper show that each unique visitor generates around 2.5 visits —  a measure that’s stayed the same for years, and that’s bedeviled papers who think readers no longer engage with them. But Harmon thinks that figure is a product of faulty measurements. He says papers’ loyal readers report they visit around 40 times a month, a figure that should make papers feel better about their connection with the audience. But if those readers visit from two computers, their monthly visits will be recorded as 20, not 40. And if they clear cookies once a week, the number of apparent visits drops to five. The effect of all this? Papers think they have many more unique visitors than they do — and engaged, repeat visitors look like multiple drive-by readers. Harmon thinks that’s a fundamental misunderstanding that has, in turn, has distorted the advertising market — big, amorphous audiences are of interest to Madison Avenue, not local businesses. And when page views get commoditized, CPMs go down.

Harmon says that publishers haven’t wanted to believe him, but now that the industry is looking aggressively at paid models, his argument is getting a new hearing.

“It’s kind of a beautiful moment for us,” he says.

In the wake of the API panel, Alan Mutter discussed why publishers need real numbers as they consider for-pay plans — a plan that assumes 15% of visitors will pay for news is going to end rather badly if a large proportion of those visitors don’t actually exist. As Harmon puts it, “using incorrect information is actively dangerous to your enterprise.” (The comments on Mutter’s post also introduced me to the term scalejacking — marketers’ obsession with big audience numbers and exposure at the expense of small audience numbers and engagement — more from Seth Godin here.)

Harmon’s inclined to agree. His research tells him the real audience is much smaller. But those readers also visit more frequently, are far more engaged and are more local than they appear when diffused amid a sea of phantoms.

“We have all been enamoured of the big number, and we’ve always suspected it was wrong,” Harmon says, adding that “as long as all we were doing was selling commodity page views, it really didn’t matter. It was a shiny gold badge. But if we really want to go paid, the shiny gold badge doesn’t work anymore.”