David Kirby HuffPost, Take 2: My Original Story was Flawed, So Here’s A Second (”Corrected”) Story That’s Still Flawed, But I Hope I Can Snow You Under Again This Time…Published June 22nd, 2008 in Autism, Child Health, Fallacious Medical Reporting, Infant Health, MMR Vaccine, Medical & Epidemiological Studies, Medical Reporting, Quacks, Vaccines
(NOTE: My original post on this topic mischaracterized the 2003 CDC vaccine investigation as an “Ecological Study,” which it was not. I am reposting this piece to reflect that information accurately, but also to point out that many of the weaknesses identified in the CDC’s data and methods apply to the published 2003 “retrospective cohort” study, as much as they do to any future “ecological” ones. I regret and apologize for the error.)
I hope I’m not getting a big ego, but I have a suspicion that Kirby read my post critiquing his story of yesterday, decided he had been confused about ecologic studies, and decided to create a new story. The damn problem is that Kirby’s new article is now even more confused and erroneous than the first one. The first sixteen paragraphs’s of Kirby’s new article are devoted to the Verstraeten et al. 2003 study that found no link between mercury in vaccines and autism, ADHD, speech delay or tics. Kirby claims that this study was a major issue in both the 2006 Report of the NIEHS Expert Panel and in the CDC Report responding to the NIEHS report.
So here’s some cutting and pasting from my previous post, to remind you of some of Kirby’s major misinterpretations of the NIEHS Report, expecially regarding the Verstraeten et al. 2003 study.
Nowhere in the 2006 report did the NIEHS panel conclude that the CDC’s 2003 thimerosal safety study was riddled with “several areas of weaknesses” that combined to “reduce the usefulness” of the study. In fact, in the NIEHS panel meeting that generated the 2006 report, the quality of the CDC’s 2003 thimerosal safety study was not even discussed. This can be seen clearly if you carefully read the NIEHS Report of the Expert Panel.
Earlier this week Epi Wonk had a long discussion with one of the Expert Panel Members (who adamantly insisted that he/she remain nameless), who confirmed three things for me:
(1) The purpose of the NIEHS Expert Panel was exactly as stated in the report:
It has been proposed that the Vaccine Safety Datalink could be used to look at the association between autistic disorder (AD) or autism spectrum disorders (ASD) by means of an ECOLOGIC ANALYSIS (emphasis mine) that would compare rates before and after the removal of thimerosal from most childhood vaccinations. To determine the feasibility and potential contribution and/or drawbacks of such a study, and to consider alternative study designs that could be conducted using the VSD database, the NIEHS convened a panel of experts…
(2) The quality of previous epidemiological studies of the association between thimerosal and autism was not discussed.
(3) The overall quality of the 2003 Verstraeten et al. study was not discussed. Indeed, in the section of the report in which the expert panel considered research panels other than ecologic analyses, which they did dismiss as riddled with several areas of weaknesses that combined to reduce the usefulness of ecologic studies, the expert panel “…recommended that further consideration be given to conducting an extension of the Verstraten study that would include additional years for follow up, would add more managed care organizations and reexamine the criteria for exclusion of births and/or take a sensitivity analyses approach to examining the impact of various exclusion criteria.”
And we still have these discrepancies between the new story and the actual CDC Report:
KIRBY: …the NIEHS had criticized CDC for failing to account for other mercury exposures, including maternal sources from flu shots and immune globulin, as well as mercury in food and the environment. CDC acknowledges this concern and recognizes this limitation, the Gerberding reply says.
ACTUAL QUOTE FROM CDC REPORT: NIEHS Finding: Difficulty in estimating cumulative exposure of child to organic mercury: The panel expressed concern that VSD adminstrative data or medical charts would not be accurate in recording or estimating a childs total mercury exposure from sources other than vaccines, such as diet, air and water. CDC Response: CDC acknowledges this concern and recognizes this limitation. In addition to administrative data and medical chart review, CDC has employed parent interviews to identify total cumulative mercury exposure from sources other than vaccines, such as diet. Often, however, parent recall, for events several years in the past, poses limitations as well.
KIRBY: “The NIEHS also questioned why CDC investigators eliminated 25% of the study population for a variety of reasons, even though this represented, “a susceptible population whose removal from the analysis might unintentionally reduce the ability to detect an effect of thimerosal.” This strict entry criteria would likely lead to an “under-ascertainment” of autism cases, the NIEHS reported. Again, this would have been an issue in the Verstraeten data. “CDC concurs,” Gerberding wrote, again noting that VSD data are “not appropriate for studying this vaccine safety topic. The data are intended for administrative purposes and may not be predictive of the outcomes studied.”
FACT: The NIEHS Expert Panel did not “question why CDC investigators eliminated 25% of the study population.” On the contrary, when discussing potential alternative designs (other than ecological studies), another possibility that generated support by the panel was an expansion of the VSD study published by Verstraten et al. The availability of several additional years of VSD data was seen as an opportunity to provide a more powerful test of any potential association between thimerosal and AD/ASD and would enable reconsideration of some aspects of the original study design (e.g., exclusion criteria) It was unclear to the panel what effect exclusion of low birth weight infants and those with congenital or severe perinatal disorders or born to mothers with serious medical problems of pregnancy had on the results of the Verstraeten et al. study; an expanded future study in which sensitivity analyses both including and excluding children with perinatal problems was recommended. The quote that begins with CDC concurs has no bearing on the Verstraeten et al. study, as implied by Kirby. Gerberding is responding to an NIEHS Expert Panel point about case ascertainment. Here is the entire quote from the CDC report: CDC responds: “CDC concurs with the recommendation that broader ICD-9 codes should be considered. The weakness further emphasizes why an ecological design is not appropriate for studying this vaccine safety topic using the VSD. The VSD data are intended adminstrative purposes and may not be predictive of the outcome studied. Because the outcomes have not been validated and considering the sensitivity of this issue, any VSD study of vaccines and autism, including a broader list of ICD-9 codes, would require chart review.”
Kirby ends his new article with the following postscript:
“This revised piece does raise two new questions, I think:
1) If the VSD is not necessarily appropriate to help determine the effect of reducing mercury levels in vaccines, are taxpayers getting their money’s worth?
2) If studies done in Denmark, Sweden and California were also “ecological” in nature, are they subject to some of the same weaknesses and limitations?”
Epi Wonk Response:
1) Neither the NEIHS Report nor the CDC Report state anywhere that the VSD is not appropriate to help determine the effect of reducing mercury levels in vaccines.
The relevant summary statements are:
(A) The NIEHS panel identified several serious problems that were judged to reduce the usefulness of an ecologic study design using the VSD to address the potential association between thimerosal and the risk of AD/ASD.
(B) “CDC concurs”, Dr. Gerberding wrote, “that conducting an ecologic analysis using VSD administrative data to address potential associations between thimerosal exposure and risk of ASD is not useful.”
(C) The NIEHS “panel identified several major strengths of the VSD to be: its ability to detect infrequent, vaccine-related adverse events of modest size; the possibility to supplement the MCO administrative data with reviews of medical records, interviews with parents and children, and additional diagnostic assessments; and the availability of demographic information about the MCO members.”
(D) “CDC agrees with the panels assessment of the strengths of the VSD Project to evaluate vaccine safety concerns. The VSD is a unique public-private collaboration that provides a model for the study of patient safety concerns by using individual-level data. In addition, CDC recognizes the tremendous value of the VSD as a national resource of expertise in vaccine safety research.
2) The NIEHS Expert Panel recommended that ecologic studies should not be done using the U.S. Vaccine Safety Datalink. Are completely different types of data from Denmark, Sweden, and California on which ecological analyses have been done subject to some of the same weaknesses and limitations? The answer is NO, but I suppose I’ll have to do a an entire instructional post on this for Mr. Kirby’s benefit.