James’s Musings

thoughts, photography, and geeky stuff
from an unrelentingly curious Silicon Valley entrepreneur

Jon Stewart’s Audience Bests NPR Listeners in Current Events Knowledge (!)

by James G. Beldock on August 19, 2008

Two years ago, I on­ly half-jok­ing­ly asked whether Fox News kills brain cells, about the re­sults of a re­mark­able study based on data gath­ered by my friend Michel Floyd‘s for­mer com­pa­ny1. (See the “amaz­ing co­in­ci­dence” fol­low-up post­ing, and Michel’s com­ments to it.) His data showed that view­ers of Fox News Channel reg­u­lar­ly scored half as well on tests re­gard­ing ba­sic facts of cur­rent events than did lis­ten­ers to National Public Radio. Of course, NPR has some­thing of a “high fa­lutin'” rep­u­ta­tion, so per­haps this is to be ex­pect­ed (al­though judg­ing from the blogosphere’s re­ac­tion to my post, it was nev­er­the­less a cause for some de­bate!). But I must ad­mit that even I was sur­prised to dis­cov­er this evening that view­ers of Jon Stewart’s “The Daily Show” beat even NPR lis­ten­ers on anal­o­gous tests!

Buried with­in the “Primary Sources” sec­tion of next month’s The Atlantic is a sec­tion ap­pro­pri­ate­ly head­ed “Seriously Funny,” re­count­ing a re­port from the Project for Excellence in Journalism at journalism.comThe journalism.com re­port sum­ma­rizes a num­ber of stud­ies by the Pew Research Center for the People and the Press, one of which of­fers the fol­low­ing sur­pris­ing com­par­a­tive cur­rent events knowl­edge scores:

Pew Study:  Knowledge Levels by News Source
source2

Your eyes are not de­ceiv­ing you: The Daily Show and Colbert Report’s view­ers ac­tu­al­ly scored high­er on this par­tic­u­lar test than did NPR lis­ten­ers. (Note: I want a mar­gin of er­ror on this mea­sure­ment, and the Pew study doesn’t iden­ti­fy one, so it’s hard to tell how mean­ing­ful this 3% dif­fer­ence is.) But there is no ques­tion that the data cor­rob­o­rate the ear­lier Knowledge Networks study: Fox News Channel yet again brings up the rear.

Pew goes a lit­tle fur­ther than the Knowledge Networks study and con­veys some of the au­di­ence de­mo­graph­ics. Of par­tic­u­lar in­ter­est, for ex­am­ple, is the fact that NPR lis­ten­ers are more like­ly to have grad­u­at­ed col­lege than reg­u­lar con­sumers of any oth­er news me­dia oth­er than ma­jor news­pa­pers’ web­sites, and that yet again Fox News Channel lags be­hind:

Pew Study:  Audience Profiles (Demographics)
source3

There are some sur­pris­ing num­bers in the­se de­mo­graph­ics, too: The Daily Show’s view­ers may know mar­gin­al­ly more about cur­rent events than lis­ten­ers to NPR, but they are sub­stan­tial­ly less like­ly to have grad­u­at­ed col­lege (on­ly about 75% as like­ly). Equally sur­pris­ing is that the Daily Show’s de­mo­graph­ic is slight­ly less like­ly to be young than reg­u­lar read­ers of ma­jor news­pa­per web­sites or Google/Yahoo! news. In oth­er words, if you’re aged 18-29, you’re most like­ly to get your news on­line.

Or, to put it an­oth­er way, if you’re over 29, why are you read­ing this? 😉

Be Sociable, Share!
  1. Michel was then CTO of Knowledge Networks and his col­league Stefan Subias, con­duct­ed by PIPA (the Program on International Policy Attitudes) and pub­lished in Political Science Quarterly []
  2. The Pew Research Center for the People and the Press, Public Knowledge of Current Affairs Little Changed by News and Information Revolutions []
  3. The Pew Research Center for the People and the Press, Public Knowledge of Current Affairs Little Changed by News and Information Revolutions []
Michel Floyd August 20, 2008 at 10:33 am

Hey James! It wasn’t actually my study, it was just run by my company. [ed note: I’ve updated the blog entry to get the facts straight, thanks Michel! –JGB] I can’t remember who the lead investigator was on it, I’ll have to look up the original.

I’ll have to dig into this more recent Pew study as well. Knowledge/awareness of current events isn’t the same as being aware of the facts. There’s no way that O’Reilly viewers are nearly as *correctly* informed as NPR or Daily Show viewers!

Michel Floyd August 20, 2008 at 2:44 pm

More on this James.

1) The principal investigator in the original PIPA/Knowledge Networks study was Steven Kull of PIPA. Knowledge Networks fielded the survey but the KN employee most directly involved was Stefan Subias.
2) The original PIPA study deal with misconceptions about the Iraq war and showed that the principal news source for the respondents was highly correlated with the number of misperceptions. For example, viewers of Fox News were much more likely to believe that Iraq had been involved in 911 than listeners of NPR (a misconception that still persists mind you!). The more recent Pew work cited in your post that was drawn on by journalism.com and The Atlantic was targeted at more general knowledge. For example: “Who is the president of Russia?” The answers to these questions are very straightforward and not as subject to ideological bias. So people who consume a lot of news are going to get these questions right more often. Intelligence and education will also correlate highly with correct answers.

If you re-ran the earlier PIPA work with current questions such as “Is Obama a muslim?” or “Is China drilling for oil off the coast of Florida?” I surmise that one would again uncover that viewers of Fox News would be more likely to get these questions wrong than viewers of The Daily Show or NPR listeners.

Paul October 13, 2009 at 10:36 am

One would be hard pressed to say that the educational system in this country is non-biased. This must, by necessity, taint the results of this survey. Assuming the average college graduate is better at assimilating and retaining knowledge, in general, then any political bias imparted by the educational system will naturally skew surveys such as this.

In other words, when measuring the efficacy and accuracy of a news agency, you can’t ignore the basic cognitive abilities of the general audience.

For example, if the average IQ of Fox’s general audience was 45 points less than the IQ of NPR’s general audience (hypothetically speaking of course) they would, naturally, perform poorly at correctly remembering any facts presented when compared to NPR’s audience.

NPR tends to attract a more educated audience (in my opinion… feel free to argue that point) so if it’s not at the top of the list with a reasonable margin it must be performing poorly in comparison.

In short, if you’re going to compare the accuracy and efficacy of a news service’s audience based on the accuracy of their audience’s recall, you must measure the cognitive abilities of the base audiences in general. You can’t assume the average NPR listener will have the same cognitive abilities as the average Network Morning Show listener.

Furthermore, this is just one issue in making such a comparison and coming to any conclusion about the news service in general. Another issue that comes to mind is audience attention level. For example, many people may have a morning news show on but may give it far less attention than those listening to an evening broadcast.

The real fallacy here is assuming causality. Surveys are fairly good at showing correlation, but are very poor at showing causality. It shows that there is a significant deviation in factual recall between listeners of The Daily Show and network morning shows, but it doesn’t show why. Jumping to any conclusions about the accuracy or efficacy of these news services is fallacious and misleading.

Comments on this entry are closed.

Previous post:

Next post: