I just found the erratum to the original Ip paper (It is mentioned after the Ip paper but not linked) and it completely corrects Table 1.

1. The original problem DeSoto had with the data was that using the standard deviations in Ip the t-test should have been significant. The problem is the blood Hg std in Table 1 are wrong, they have lost the first digit and should be 15.65 and 12.49. On the same topic the std for age are obviously ridiculous and should be 2.8 and 3.5 instead of 0.2 and 0.4. The text claims a rage of 4-11 years, which is clearly incompatible with the published std. In fact the age range is 3-16. I didn’t check the rest of the table.

2. For the t-test I get a p-value of 0.057 compared to Ip 0.15. Taking logs (which seems correct based on residuals) reduces this to 0.047. Removing the two outliers gives a p-value of 0.018 for unlogged data.

3. Using logistic I get 0.06 for regression on blood Hg, 0.02 with the outliers removed (close enough to DeSoto) and 0.05 with log blood Hg. Removing the outliers doesn’t seem very conservative as DeSoto claims and this should have been reported. After log transformation they don’t look like outliers any more.

4. Ip based the desicion to not include age and gender in the model because they weren’t significantly different between the groups. Not a valid reason but they don’t make much difference anyway. Logistic regression with log blood Hg, age and sex has a p value of 0.03 for log blood Hg but age and sex are not significant.

5. Still leaves the problem that blood Hg is obviously censored at 5 and the analysis should take this into account, although it probably wont make much difference. A topic for Advanced Data Analysis.

6. Ignoring the censoring a statistician could sensibly fit a logistic with log blood Hg, age and sex and the resulting p value for log blood Hg would be 0.03. Significant but not hugely. So while the statistical analysis isn’t optimal it doesn’t really change the conclusions. Still leaves the problem of the bias which is probably much more important. In addition to those already mentioned is the assumption that current blood mercury is an indication of blood mercury prior to development of autism.

That should say “asymptotically normal from the CLT PLUS Slutsky’s theorem” in the 6th paragraph. I don’t know why the “PLUS” went away (it’s in the original copy of the reply that I wrote in notepad and pasted into the window).

(but with an actual plus symbol where I just typed PLUS)

If you wish, I’m happy if you just make sure the PLUS is present in the original message and delete these two followups.

Even in the case of ordinary regression, the X-variables don’t have a specific distributional assumption - the model is conditioned on them. Only the Y-variable does and then only its conditional distribution. The fact remains that given the hypothesis that mercury causes autism, mercury level is not the response, but whether or not you were diagnosed with autism, and so the distribution of mercury levels is not an issue unless you are trying to do some kind of inverse regression..

[Actually, I some further issues with some of your criticisms of their paper, but I think I should stick to the main points.]

I’m not certain what the source of the “F-test” figure was, but I do see one possibility. Or rather, I see two different possibilities, but they’re just the same thing looked at two different ways; I will discuss one of them.

Be aware that this is mere conjecture - you’d have to ask the authors to be sure.

In GLMs, some people call the ratio of a coefficient to its standard error as a “t-statistic”. In ordinary regression, the square of this t-statistic has an F-distribution.

Note that in GLMs this “t-statistic” doesn’t actually have a t-distribution (it’s still “asymptotically normal from the CLT + Slutsky’s theorem”). With large d.f. it doesn’t matter though, since we’re just using the normal tables anyway, so you end up in the right place whatever you call it, and calling it a t-statistic has the advantage of helping us understand what thing we’re looking at. With large d.f., if you happen to look up t-tables it won’t impact your conclusions, so it’s more a matter of poor terminology than bad practice.

[Some people argue that the numerator is asymptotically normal and the denominator is asymptotically the square root of a chi-squared on its df and “hence it’s asymptotically t”. This argument has two different flaws. It may or may not be asymptotically t but that argument is not sufficient to establish that it is.]

Anyway, if you square that standardized coefficient, you might call it an F-statistic by analogy. Both the squared statistic and the F-table asymptotically approach chi-square (in the same way that the original statistic and the t-table both approach normality), so again, the conclusions should be correct.

If that’s what they did, their terminology may be a little sloppy but the correct impression is generated; if it was done with the aim of presenting information more familiar to users of ordinary regression, I wouldn’t have a big problem with it.

But again — thank you very much for your comment — it woke me up to my error.

Thank you also to Ken; I should have realized my mistake when he made similar comments.

efrique: Here’s a question that still has Ken and I partially stumped: Where does the F-test come from in DeSoto & Hitlan’s logistic regression results, which I quote in my article? They must have done some sort of least squares comparison of means to get an F-test.

I had noticed previously that the blood mercury difference of autistic vs. non-autistic females in this data set was greater than that of males. I’m not sure if there’s statistical significance on that.