Unless you’re extremely vigilant and/or organized, it really is easy to miss important information. Just today I came cross a June 21st post on Ben Goldacre’s Bad Science blog on a May 27th PLoS Medicine article entitled, “How Do US Journalists Cover Treatments, Tests, Products, and Procedures? An Evaluation of 500 Stories.”
I also discovered a new blog (started in June), a healthy distrust, which is devoted primarily to issues like: science in the media, scientists versus journalists, accuracy of press coverage of science, the use of press releases by journalists (instead of actually reading the study), the reporting of unpublished research in the press (e.g., poster presentations from conferences [my example]), quoting the public as a source in scientific stories, and using quacks (my term) as sources to balance the story. Frankly, my brief description doesn’t do the blog justice. See: “The biggish picture - or why this blog exists.”
The author of the PLoS Medicine article is Gary Schwitzer, Associate Professor at the University of Minnesota School of Journalism and Mass Communication, Publisher of Health News Review, and blogger of the Schwitzer health news blog. The article is about a US Web site project, HealthNewsReview.org, which evaluates and grades health news coverage, notifying journalists of their grades. Schwitzer begins the article by briefly describing two similar projects:
(1) The Australian Media Doctor Web site, which monitors the health news coverage of 13 Australian news organizations.
(2) The Canadian Media Doctor Web site, which evaluates health news coverage by 12 Canadian news organizations.
HealthNewsReview.org monitors news coverage by the top 50 most widely circulated newspapers in the US; the most widely used wire service, the Associated Press; and the three leading newsweekly magazines — TIME, Newsweek, and U.S. News & World Report. Each weekday they watch the morning and evening newscasts of the three most watched television networks — ABC, CBS, and NBC.
SUMMARY POINTS OF THE ARTICLE
- The daily delivery of news stories about new treatments, tests, products, and procedures may have a profound — and perhaps harmful — impact on health care consumers.
- Health News Review evaluates and grades US health news coverage, notifying journalists of their grades.
- After almost two years and 500 stories, the project has found that journalists usually fail to discuss costs, the quality of the evidence, the existence of alternative options, and the absolute magnitude of potential benefits and harms.
- Reporters and writers have been receptive to the feedback; editors and managers must be reached if change is to occur.
- Time (to research stories), space (in publications and broadcasts), and training of journalists can provide solutions to many of the journalistic shortcomings identified by the project.
In order to be eligible for review, a story must include a claim of efficacy or safety in a health care product or procedure (drug, device, diagnostic or screening test, surgical procedure, dietary recommendation, vitamin, supplement). The rating instrument used includes ten criteria addessed by the Association of Health Care Journalists’ Statement of Principles (A Statement of Principles for Health Care Journalists by Gary Schwitzer. American Journal of Bioethics 2004; 4: W9 - W13):
1. Adequately discusses costs.
2. Quantifies benefits.
3. Adequately explains and quantifies potential harms.
4. Compares the new idea with existing alternatives.
5. Seeks out independent sources and discloses potential conflicts of interest.
6. Avoids disease mongering.
(Journalists should “avoid promulgating the medicalization of normal states of or variations in health (e.g., baldness, menstruation, short stature, etc.). We also try to educate journalists about surrogate endpoints and about how risk factors are not diseases. With this criterion, we also remind them not to exaggerate the prevalence or incidence of a condition.”)
7. Reviews the study methodology or the quality of the evidence.
8. Establishes the true novelty of the idea.
9. Establishes the availability of the product or procedure.
10. Appears not to rely solely or largely on a news release.
In their evaluation of 500 US health news stories over 22 months, between 62% - 77% of stories failed to adequately address costs, harms, benefits, the quality of the evidence, and the existence of other options when covering health care products and procedures. Only 38% of stories were rated satisfactory for putting the intervention under discussion into the context of existing alternative options. Of the first 500 stories reviewed, 41 (8%) received their highest scores. They appear online at http://www.healthnewsreview.org/review/by_rating.php?rating=5. (As of today, the 22nd of July 2008, this has been updated to 73 five-star reviews out of 615 stories.)
Gary Schwitzer concludes the article by hoping that HealthNewsReview.org’s “evaluation of health news will lead news organizations — and all who engage in the dissemination of health news and information — to reevaluate their practices to better serve a more informed health care consumer population.”
I’d like to end today’s post by once again welcoming a healthy distrust to the blogosphere.