Back in 1987 right after I’d gotten out of law school and passed the bar one of the first things I worked on was objecting to a request by plaintiff’s counsel for all of the death certificates gathered by the University of Texas School of Public Health for its study of our client’s refinery. Plaintiffs’ retained epidemiologist wanted to do his own "unbiased" study, he claimed.
"Have you heard of the Texas Sharpshooter?", asked the school’s biostatistician when I met with him for the first time. No, I hadn’t. But I would soon understand that when you get to draw your target after you see where your shots have struck, your accuracy improves dramatically. And that was exactly what happened when plaintiffs’ expert did his study. Suddenly big risks appeared where none had been found by other investigators.
The problem, of course, was that the expert’s hypothesis was generated by the litigation, and not by scientific curiosity. He knew who had developed leukemia and so, after reviewing payroll records and depositions, honed in on the inevitable commonalities among the afflicted workers. Thus he discovered what tasks must, in his mind, have lead to the causative exposures; and, sure enough, his unpublished study showed that that there was a big increase in leukemia among those with work histories like our plaintiffs.
Yet even in Beaumont, even in the bad old days, the judge who usually gave us fits (admitting the Sumner Simpson papers against defendants who’d never heard of them, much less participated in the alleged conspiracy) recognized the fallacy embedded in the claims of plaintiffs’ expert and excluded his ex post epi study (though he had let plaintiffs get the death certificates). You can always find something that events have in common after they’ve occurred, the court held. The trick, and the real test of a scientific theory, is whether it can predict those events before the test is run (or the Texas sharpshooter’s shots are fired).
According to the scientific method a sound test is one designed to test a theory’s predictions and not just to verify the investigator’s preconceived notions. Sadly, that distinction was overlooked by the New York appellate court in Nonnon II. Nevertheless, all surely cannot be lost when such things go better in Illinois. The very point missed by the court in Nonnon II was seized upon by an Illinois trial court. It examined the findings of the same expert doing the very same sort of ex post epi study who arrived at the same conclusion (which again flew in the face of objective, blinded studies done by the state) and, deciding enough was enough, threw out plaintiffs’ claims. Looking at the two rulings it appears we haven’t come very far in the last 25 years. On one side is the emotive power of claims fanned by fears of childhood leukemia clusters and on the other the cold hard facts and a whole lot of uncertainty. Not much of anything has changed I suppose.
Note to readers: We’re coming up on 800 posts and I’ve just now learned that I routinely fail to check the "Send Notification" box upon posting. Thus those of you who’ve signed up haven’t gotten notifications of the vast majority of our musings. I apologize and promise to check the box henceforth. h/t Laura K. Deen.