All three firms have failings

by Dan Miller

      First there was Media Metrix. Then came Nielsen//NetRatings. Now PC Data is on the scene, as of late last month. Three firms to argue over which delivers the most accurate view of online traffic and usage.
      Media buyers and planners may be forgiven for wondering, sometimes, if any of them do.
      Take Lycos's well-publicized victory over portal king Yahoo! revealed by Media Metrix figures at the end of last month. Lycos has proclaimed itself the ''most-visited Internet hub'' ever since.
      But for Art Tatnell, senior vice president of media information and technologies at Bates USA, no amount of boasting by Lycos could convince him the numbers were anything but ''absolutely meaningless.'' The web, he says, is most effective for building better relationships with consumers. This places much more emphasis on the type of audience than sheer size.
      And then there's the larger problem of measurement standards. There are none. In their absence, ratings companies are left to their own sense of propriety. Media Metrix may, for instance, decide not to add acquisitions to a web company's numbers until deals are formally closed. This is why Lycos was able to overtake Yahoo!; GeoCities and Broadcast.com, recently acquired, were not included because the deals weren't finished. Add them in and Yahoo! is ahead of Lycos by many many millions of visitors.
      Long before anyone does that, however, the web sites themselves are happy to take positive-sounding numbers and run with them. Jupiter Communications analyst Marissa Gluck warns people to cast a skeptical eye. ''These announcements are not for media buyers, they're for investors,'' she says.
      All three ratings rivals derive their numbers from volunteers who install software on their PCs to record and report their web usage. But that's where similarities end. Both Nielsen and Media Metrix recruit by calling a random sampling of U.S. residents; PC Data seeks volunteers via advertising and by putting copies of its 99-cent software on retail shelves (users receive a $5.00 rebate).
      The established players are not impressed with the upstart's methodology. ''PC Data's recruiting method is like posting a billboard above the Lincoln Tunnel that says 'Come on in!,' says Doug McFarland, senior vice president of sales and account development for Media Metrix. ''The point is, it's not random, it's self-selected. If it's not random, it's not good research.''
      As it turns out, the composition of all three measurement firms' survey pools is in dispute. All three claim to measure both home and business users. But many companies object to having the necessary software installed on office PCs. Given that some 50 percent of web traffic occurs during business hours, that restriction is enough to skew the ratings in favor of home surfers.
      Then there's the question of demographics. All three services collect demographic data from their recruits. But while Nielsen and Media Metrix select recruits based on their demographics, PC Data goes at it the other way: they recruit their users, then weight their results according to an IntelliQuest profile of who's online--the percentage of men vs. women, the number of users in California vs. Utah. ''We'll sign up anyone,'' says Ann Stephens, president of PC Data, ''but we'll weight them differently.''
      Once they have their recruits signed up and profiled, all three services proceed to measure their subjects' surfing habits. But, again, there are specific differences. Media Metrix claims to record all online activity--including use of America Online--not just web surfing with a browser. McFarland claims this gives him more realistic numbers. His competitors ''cannot measure the largest financial [sites], the largest news [sites], the largest sports [sites] online, because they cannot measure America Online.''
      None of these services can claim to measure traffic on every one of the 43 million-plus web sites out there, but they differ spectacularly on what fraction each claims it can report on. Media Metrix asserts it can reliably report on traffic at the top 18,000 sites, PC Data on 3,000. But, says Sally Blodgett, of Nielsen, ''There's no way they can do that reliably.'' Her company reports regularly on 691 sites.
      Why so few? ''We need to have a certain percentage of our sample visiting a given site before we'll report on it,'' explains Blodgett. ''Reporting on more is going to be statistically suspect. We have a very high reporting threshold.''
      Media Metrix boasts some 350 clients; Nielsen//Net Ratings, more recently launched, has 100, but includes among them some heavy-hitters: Yahoo!, Amazon, Microsoft, Procter & Gamble, and, a recent addition, AOL.
      Newcomer PC Data doesn't have comparable client numbers yet. But if pricing is any indicator, it should soon have a corner on the low-end of the market. While Nielsen and Media Metrix charge in the five- to six-figure range for their data, PC Data is taking the budget route: ''If you want everything we do, and you don't subscribe to the other services, we'll charge you $7,500 a year,'' Ann Stephens says. ''That's about a tenth of the other services.''
      So who's going to win the race to become the Nielsen of the net?
      ''If this follows the traditional ratings model, there is not much room for three contenders,'' says Rex Briggs, executive vice president, Millward Brown Interactive, an online marketing research company. ''[But] calling out a leader at this point is like judging a pack of marathon runners in the first tenth of a mile.''
      All three face questions of credibility from media buyers raised on TV ratings, he says.
      ''Do I trust their ratings to tell me which are the big sites? Yes. Would I trust them to develop a media plan? Never. The whole problem is that ratings are a broadcast measurement yet web advertising is bought as a 'narrowcast.' What does it matter that ABC is bigger or smaller than NBC if both can deliver you 500,000 impressions?''