Re: Carrying a big stick #2 Arb

[Garrett Wollman <wollman>]

<<On Fri, 22 Feb 2002 01:02:18 -0500, "A. Joseph Ross" <lawyer@attorneyross.com> said:

> Size of the sample isn't as important as the sample being representative. 
> Remember the story about the North American Review.  It took a very large 
> poll of its readers and concluded that Landon was going to defeat 
> Roosevelt in 1936.  Many people at the time made fun of the small sample 
> used by the Gallup organization, but Gallup, with its much smaller 
> sample, predicted the Roosevelt landslide.

It is far, far easier to create a statistically-valid survey to
determine a ``leading preference''.  It is extraordinarily difficult
to create a survey that can accurately reflect preferences #2 through
#25.  It is orders of magnitude more difficult to create a survey that
will estimate the precise number of left-handed African-American women
over age 35 with household incomes of at least $50,000 living in the
781 area code who listen to the seventh-ranked station -- yet this is
the sort of information that is supposedly Arbitron's stock-in-trade.
Many people feel that it does a fairly poor job of this.

Looked at another way: I can pick five people completely at random,
and I have at least an unbiased sample.  However, even five people,
taken together, are exceedingly unlikely to express the complete
spectrum of preferences.

Arbitron's methodology has also come under question.  There are good
reasons to believe that neither diaries nor telephone surveys
accurately reflect actual listening habits, because people do not care
as much about which radio station they listen to as the stations care
about which people are listening to them.  One company attempted to
start a new radio ratings service, oriented towards commuters (one
group particularly unlikely to fill in diaries accurately, since they
are concentrating on driving and not on the radio), which would
monitor the local oscillator leakage of passing car radios at busy
intersections throughout the survey area.  I have not heard of what
became of this company.

-GAWollman