For many years, broadcasters and knowledgeable advertising agencies have been skeptical about the accuracy of “diary” reports by Arbitron. Historically, these have been many cases in just about every market, including Seattle, of inconsistencies and fluctuations in the reporting of radio listening. The reason most everyone believed, was the diary methodology.
With the diary, people were chosen by computer in each market, based on a cross-section of age, sex, location and other factors. On paper, the sample was perfect. But, in reality, it was flawed.
To compile a report (commonly referred to as and ARB). Arbitron relied 100% on the participants filling out the diary and retiring it to them. Many did not do this and sometimes the diaries were heavily skewed to certain geographic areas (e.g., Tacoma).
Also, Arbitron often didn’t get enough diaries back for specific age groups, or received too many for one or more age groups. When this happened, Arbitron did what it called “weighting.” This involved taking the flawed sample and “weighting” it up or down to arrive at the planned large oil for each age cell. In short, the took the flaw and multiplied it, usually creating some unrealistic radio-audience estimates.
Another diary flaw was the idea that participants would fill out the diary on a daily basis for a one-week period. It was widely believed that many waited until their participation period was done and then try to remember when they listened and to what stations. Obviously, most people can’t accurately remember over a one-week period, so it was much simpler to report that they listened to one station for long periods of time.
Radio programmers and knowledgeable radio people believed that most radio listeners do listen to one station a great deal, but they often change stations. The diary method really did not document that.
Another flaw in the diary was that the participant sample changed every week, thus every week different people were asked to record their listening. There was no way the diaries could record changes in listening habits, especially if a station changed its format. With the change, did the audience go up or down?
Enter the PPMs (Personal People Meters), a cell-phone-sized electronic device that participants are asked to carry with them at all times. It records every station the participant actually is exposed to, rather than what a participant said he/she was listening to with the diary.
I say “exposed to” because, for example, if a person carrying a PPM is in a store that has a radio station playing over its sound system, the device will record that station. The PPM, in fact, will record any station sound as long as it’s audible, which is one big flaw with that methodology.
Nevertheless, there are several obvious advantages to the PPM. First, it accurately records which station each participant actually listens to. Second, no memory is required; the device automatically records all data and transmits it once a day to the control center. Finally, the PPM can instantly record changes in listening habits, if a participant changes stations or begin listening to a new station.
The biggest question with the PPM reports is the sample size. The last diary report in Seattle had 3,702 participants. The PPM reports are based on a sample about one-third that size. Arbitron believes the PPM sample, although smaller, projects a more accurate picture of radio listening. Experts believe it will take a year or more to track the PPM reports to determine whether they truly give a more accurate reading of radio listening than the diary method.
So, how have the PPMs change the reporting of Seattle radio listening? Drastically! Some of the radio stations previously shown with large audiences have dropped substantially, and others have jumped dramatically.
The PPM report for April-June ’09 documents that total radio listening is much greater than reported previously with the diary method. For example, in the Winter ’09 Arbitron diary report, only two stations showed a total cumulative audience of 400,000 or more. In the new PPM report, 17 stations went reported with 400,000+ – a huge jump! Eight stations showed 600,000 or more listeners and two had more the 800,000.
Putting the total radio listenership in the new PPM report into perspective, The Seattle Times has a daily circulation of 209,396 according to the Audit Bureau of Circulation. Seven radio stations have more daily listeners than the total Times circulation.
There also were winners and losers. The biggest winners were KJR-FM, KRWM-FM, KJAQ-FM, KPLZ-FM and KOMO-AM/PM. Stations like KCMS-FM, KZOK-FM and KISW-FM were fairly consistent from the diary to the PPM report.
The big surprise losers were KIRO-FM, KTTH-FM and KVI-AM. In the Winter diary, KIRO-AM (then “news/talk”) was ranked 3rd in the market with Adults 25-54, but in the new PPM report, KIRO-FM fell to 20th. In the Winger diary, KTTH-AM ranked a respectable 9th, but fell to 21st in the PPMs. And KVI-AM fell from 20th to 26th.
Once of the biggest changes in total radio listening between the diary and the PPM reports is the substantial difference shown in when people are listening to the radio. For many years, it was widely believed–and documented by the diary reports–that morning drive was by far the most-listened-to part of the day, with mid-days ranking second and afternoon drive third in total listening. No more!
In the PPM report, afternoon drive is the most-listened-to part of the day, with mid-days a close second. This is a whopping change in radio listenership patterns and shatters the myth that most people listen to the radio in the morning. The largest listenership hours 12+ are 3pm, 2pm, 4pm, 7am, noon, 5pm and 8am, in that order. Listening on weekends also has increased substantially in the new PPM report.
Are the PPMs more accurate? Most knowledgeable broadcasters and statisticians believe so. The new PPM methodology isn’t perfect. There are flaws in the report and I’m sure Arbitron will make many adjustments in the months and years to come.
Stations that have fallen are aggressively challenging or have filed suit against the new PPM reports. But like it or not, they’re here to stay and the only industry-recognized source of radio-listening estimates.