Two years ago, I only half-jokingly asked whether Fox News kills brain cells, about the results of a remarkable study based on data gathered by my friend Michel Floyd’s former company1. (See the “amazing coincidence” follow-up posting, and Michel’s comments to it.) His data showed that viewers of Fox News Channel regularly scored half as well on tests regarding basic facts of current events than did listeners to National Public Radio. Of course, NPR has something of a “high falutin’” reputation, so perhaps this is to be expected (although judging from the blogosphere’s reaction to my post, it was nevertheless a cause for some debate!). But I must admit that even I was surprised to discover this evening that viewers of Jon Stewart’s “The Daily Show” beat even NPR listeners on analogous tests!
Buried within the “Primary Sources” section of next month’s The Atlantic is a section appropriately headed “Seriously Funny,” recounting a report from the Project for Excellence in Journalism at journalism.com. The journalism.com report summarizes a number of studies by the Pew Research Center for the People and the Press, one of which offers the following surprising comparative current events knowledge scores:
source2
Your eyes are not deceiving you: The Daily Show and Colbert Report’s viewers actually scored higher on this particular test than did NPR listeners. (Note: I want a margin of error on this measurement, and the Pew study doesn’t identify one, so it’s hard to tell how meaningful this 3% difference is.) But there is no question that the data corroborate the earlier Knowledge Networks study: Fox News Channel yet again brings up the rear.
Pew goes a little further than the Knowledge Networks study and conveys some of the audience demographics. Of particular interest, for example, is the fact that NPR listeners are more likely to have graduated college than regular consumers of any other news media other than major newspapers’ websites, and that yet again Fox News Channel lags behind:
There are some surprising numbers in these demographics, too: The Daily Show’s viewers may know marginally more about current events than listeners to NPR, but they are substantially less likely to have graduated college (only about 75% as likely). Equally surprising is that the Daily Show’s demographic is slightly less likely to be young than regular readers of major newspaper websites or Google/Yahoo! news. In other words, if you’re aged 18 – 29, you’re most likely to get your news online.
Or, to put it another way, if you’re over 29, why are you reading this? 
- Michel was then CTO of Knowledge Networks and his colleague Stefan Subias, conducted by PIPA (the Program on International Policy Attitudes) and published in Political Science Quarterly [↩]
- The Pew Research Center for the People and the Press, Public Knowledge of Current Affairs Little Changed by News and Information Revolutions [↩]
- The Pew Research Center for the People and the Press, Public Knowledge of Current Affairs Little Changed by News and Information Revolutions [↩]
Hey James! It wasn’t actually my study, it was just run by my company. [ed note: I've updated the blog entry to get the facts straight, thanks Michel! --JGB] I can’t remember who the lead investigator was on it, I’ll have to look up the original.
I’ll have to dig into this more recent Pew study as well. Knowledge/awareness of current events isn’t the same as being aware of the facts. There’s no way that O’Reilly viewers are nearly as *correctly* informed as NPR or Daily Show viewers!
More on this James.
1) The principal investigator in the original PIPA/Knowledge Networks study was Steven Kull of PIPA. Knowledge Networks fielded the survey but the KN employee most directly involved was Stefan Subias.
2) The original PIPA study deal with misconceptions about the Iraq war and showed that the principal news source for the respondents was highly correlated with the number of misperceptions. For example, viewers of Fox News were much more likely to believe that Iraq had been involved in 911 than listeners of NPR (a misconception that still persists mind you!). The more recent Pew work cited in your post that was drawn on by journalism.com and The Atlantic was targeted at more general knowledge. For example: “Who is the president of Russia?” The answers to these questions are very straightforward and not as subject to ideological bias. So people who consume a lot of news are going to get these questions right more often. Intelligence and education will also correlate highly with correct answers.
If you re-ran the earlier PIPA work with current questions such as “Is Obama a muslim?” or “Is China drilling for oil off the coast of Florida?” I surmise that one would again uncover that viewers of Fox News would be more likely to get these questions wrong than viewers of The Daily Show or NPR listeners.
One would be hard pressed to say that the educational system in this country is non-biased. This must, by necessity, taint the results of this survey. Assuming the average college graduate is better at assimilating and retaining knowledge, in general, then any political bias imparted by the educational system will naturally skew surveys such as this.
In other words, when measuring the efficacy and accuracy of a news agency, you can’t ignore the basic cognitive abilities of the general audience.
For example, if the average IQ of Fox’s general audience was 45 points less than the IQ of NPR’s general audience (hypothetically speaking of course) they would, naturally, perform poorly at correctly remembering any facts presented when compared to NPR’s audience.
NPR tends to attract a more educated audience (in my opinion… feel free to argue that point) so if it’s not at the top of the list with a reasonable margin it must be performing poorly in comparison.
In short, if you’re going to compare the accuracy and efficacy of a news service’s audience based on the accuracy of their audience’s recall, you must measure the cognitive abilities of the base audiences in general. You can’t assume the average NPR listener will have the same cognitive abilities as the average Network Morning Show listener.
Furthermore, this is just one issue in making such a comparison and coming to any conclusion about the news service in general. Another issue that comes to mind is audience attention level. For example, many people may have a morning news show on but may give it far less attention than those listening to an evening broadcast.
The real fallacy here is assuming causality. Surveys are fairly good at showing correlation, but are very poor at showing causality. It shows that there is a significant deviation in factual recall between listeners of The Daily Show and network morning shows, but it doesn’t show why. Jumping to any conclusions about the accuracy or efficacy of these news services is fallacious and misleading.
Comments on this entry are closed.