Radio
The real flaw with Arbitron's PPM
By Mike Stern
Nov 5, 2009 - 8:11:36 AM

Arbitron’s Portable People Meter is changing how radio listening is measured, and in so doing, it's stirred up a great hullabaloo amid charges that the new electronic system undercounts minorities, to the disadvantage of minority-owned radio stations.

That's the issue that's gotten a slew of headlines in the past year and some, virtually since the first day the PPM rolled out.

The fact is, the PPM, though greatly welcomed by media buyers in particular, is flawed, but not in the way minority station owners charge.

Moreover, the real flaws in the PPM may not be as easy to fix. Indeed, if you listen to other critics of the new system, making it into a truly accurate means of measuring radio listenership could well be cost-prohibitive.

The flaw in itself is quite simple. The size of the sample--the people Arbitron has recruited to wear the PPM-- is too small, critics say way too small.

In a sense, the PPM undercounts not just minorities but everyone.

The data itself is accurate, but there's simply not enough of it to always accurately reflect who's listening to what stations. And of course that's exactly the shortcoming of the old diary system the PPM was developed to overcome.

Let's take a market, Philadelphia for example.

On the face of it, the PPM sample looks comparable to the old diary sample it replaced.

Under the old system, Arbitron would place approximately 4,600 one-week diaries in the market during a 12-week survey, resulting in 32,200 days of observation. With PPM, listening is recorded using approximately 1,530 meters, which equate to 10,710 days of observation weekly, or 42,840 days in a PPM monthly report. That's approximately the same amount of data tabulated in the old quarterly diary report.

So where does the problem come in?

When that sample data is sliced and diced by demographics, ethnicity and geography. When the data is broken down in that way, there's simply too little data to get an accurate reflection of a particular subset.

As Rob Balon, CEO of Benchmark, which specializes in audience research, puts it, “The 6-plus number is valid because it includes everybody. But every time you parse the sample down you need to have enough people so even the smallest cells hold up statistically.”

In San Francisco, where some 2,000 meters are in use, the data generated for women age 35-49 may be based on just a handful of people.

What those few women listen to may well reflect what all women 35-49 listen to in that market, but there's no certainty, and you can expect a media buyer to place clients' money based on that data.

And of course the risk is that much higher that some member of that group has totally oddball listening habits. The result is a skewed profile of that subset of listeners.

Jay Guyther, a partner at ROI Media Solutions, which consults companies such as CBS Radio and Spanish Broadcasting Systems on PPM strategies, offers the example of the 20-year-old white-male who listens to a gospel station.

The solution, of course, would be to expand the number of meters in every market.

The problem is that, unlike diaries, the PPM system is expensive. The technology itself is more expensive, and whereas they dairies could be mailed, the PPM has to be installed in the sample members’ homes.

As it is, stations pay much higher rates for the PPM, and those rates would shoot up again if sample sizes were increased. They're unhappy with what they are paying now.

Says Mickey Luckoff, vice president and general manager of two Citadel Broadcasting stations in San Francisco: “I’m paying over $1 million a year for two AM stations for data we can’t use.”

Arbitron recently announced plans for a 10 percent sample increase in PPM markets, but critics say that's not nearly enough to remedy the problem. Balon and Guyther say the sample size would need to be increased somewhere between four and eight times the current size.

And of course, there's an irony to the PPM debate.

It's that for all these flaws, the PPM is a giant step forward. The data is a timelier and more accurate reflection of consumer listening habits, which is a boon to media buyers. And it also puts radio on a more equal footing with other media, which helps stations.

And it's still a huge advance over paper diaries. “If you can take an unemotional approach to this, you realize that it’s not a perfect system but it’s much more reliable than what we had,” says Guyther of ROI Media Solutions.