“Scientists have more faith in Wikipedia than national print media”

That’s one of the takeaways from a recent poll of nearly 1000 toxicologists when they were asked  how various media outlets cover their specialty: the representation to the public of chemical risks. (The poll was conducted by STATS, The Center for Health and Risk Communication at George Mason University, and the Society of Toxicology)

Given the common lament that Wikipedia shuns “experts,” and information is produced by people “off the street,” the results are intriguing when you look at the numbers for other professional and “mainstream” media outlets. From the report synopsis:

WebMD and Wikipedia were seen as significantly more accurate in the way they presented chemical risk than any other media source.

·         56% say WebMD accurately portrays chemical risks

·         45% say Wikipedia accurately portrays chemical risks

·         By contrast, no more than 15% say that leading national newspapers, news magazines, and television networks accurately portray chemical risks.

·         Over 80% say that leading national newspapers, news magazines, and television networks overstate chemical risks

[...]

…only 15 percent described similar coverage in the national print media (i.e., the New York Times, Washington Post, and Wall Street Journal) as accurate. This figure dropped to 6 percent for USA Today and 5 percent for broadcast network news.

At a press conference at the National Press Club to release the preliminary results of the study, Dr. S. Robert Lichter, who conducted the survey described the Wikipedia finding as an indictment of the mainstream media – ” it’s disturbing that someone off the street seemingly can do a better job than the media.”

I’d take issue with the fact that Wikipedia is simply the product of random person “off the street,” but it is a real shift in what we consider authority and how reliable information can be produced.

Even the best performer, WebMD, gained the approval of only about half the toxicologists who were surveyed which should be a bit surprising in itself. My (full disclosure: unpaid, uncompensated) commentary as it appeared in the report:

“This reminds me of the Nature study [link] that was done in December 2005 where it found that on average, Britannica had 3 errors per article, and Wikipedia had 4 errors,” Lih says by email. “It was surprising because Wikipedia did much better than expected, given its foreign work process and Britannica did much worse. People had presumed a certain level of accuracy from Britannica’s reputation, and it was knocked down from that pedestal. To me the WebMD and Wikipedia results here are similar – they’re much closer than what one would expect. Wikipedia doing better, WebMD doing worse.”

But perhaps the most interesting part was not WebMD, but that the daily professional print media came up so short in the eyes of these experts. It seems to reinforce the old adage: “Journalists do a pretty good job of covering things, except for subjects in which you’re knowledgeable.”

The commentary for Columbia Journalism Review contributor Alissa Quart was insightful about why the MSM approach (reporting science as a storyteller for the masses) is perhaps systemically flawed:

“Journalists fall into storylines, because that’s how we write. There are three narratives, that we use, which can make us great but also get us into trouble – one narrative to please our editors, one to please our readers, and one which leans toward our sources, because we identify with them. WebMD and Wikipedia contributors are disconnected from most of those narratives – maybe they are trying to please certain readers, but they aren’t ‘the reader.’ Their model of knowledge doesn’t ask for stories, or sentiment or people.

This is a really good observation that meshed well with my views about the role of public relations and the dangerous media narrative driving scientific reporting. Quart and I arrived at the same conclusion.

In short, argument trumps aesthetics. Lih, an engineer by education, concurs. The clash of narratives “also says something about motivation, in that the mainstream press will be driven by reports, PR bring shoved at them, and also the market and the desire by editors (in a top-down manner) to demand reporters find a story in the latest research, even if in the greater context of the field, it doesn’t warrant so much attention. In that sense, Wikipedia’s motivations are different, in that the ‘crowd’ helps moderate and even dampen the type of ‘recentism’ that is so pervasive in news coverage.”

The overall summary can be found at the Stats.org site, or you can view the full PDF.