The United States Court of Appeals, First Circuit, has delivered an opinion about the duty borne by trial courts to keep out flawed analyses by credentialed experts that is as keen in its reasoning as it is sharp in its prose. It’s "Samaan v. St. Joseph Hospital".
Have you ever despaired of explaining that a 50% increased risk doesn’t mean that half of those exposed are going to come down with the ailment associated with the elevated risk? Or that a big increase in a tiny risk rarely has anything to say about causation? Take a copy of Samaan with you the next time it comes up and have the following to hand:
" When a person’s chances of a better outcome are 50% greater with treatment (relative to the chances of those who were not treated), that is not the same as a person having a greater than 50% chance of experiencing the better outcome with treatment. The latter meets the required standard for causation; the former does not.
To illustrate, suppose that studies have shown that 10 out of a group of 100 people who do not eat bananas will die of cancer, as compared to 15 out of a group of 100 who do eat bananas. The banana-eating group would have an odds ratio of 1.5 or a 50% greater chance of getting cancer than those who eschew bananas. But this is a far cry from showing that a person who eats bananas is more likely than not to get cancer. Even if we were to look only at the fifteen persons in the banana-eating group who did get cancer, it would not be likely that any particular person in that cohort got it from the consumption of bananas. Correlation is not causation, and a substantial number of persons with cancer within the banana-eating group would in all probability have contracted the disease whether or not they ate bananas."
And then there’s this:
"Many aspects of science are a mystery to laymen without the aid of experts. In the world of the blind, the one-eyed man is king; and Daubert relevancy is the sentry that guards against the tyranny of experts. As the gatekeeper, the trial judge has the duty to insulate the jury from expert testimony when reliance on authoritative studies and methods threatens to mask the lack of an adequate fit. An expert might be able to testify on the phases of the moon to prove that it was dark by a particular time, but he could not offer the same testimony to prove that a person was likely to act in an unusual manner on that night."
But best of all and a bit earlier on in the opinion the court makes it clear that evidence, rather than mere conjecture, is required before a citizen is relieved of his property:
"The reliable foundation requirement necessitates an inquiry into the methodology and the basis for an expert’s opinion. To perform the required analysis, the district court must consider a number of factors, including but not limited to "the verifiability of the expert’s theory or technique, the error rate inherent therein, whether the theory or technique has been published and/or subjected to peer review, and its level of acceptance within the scientific community." Ruiz-Troche, 161 F.3d at 81. Given the nature of this analysis, the expert’s methodology is commonly the "central focus of a Daubert inquiry." Id.
The second requirement has attracted less attention. This requirement seeks to ensure that there is an adequate fit between the expert’s methods and his conclusions. See Daubert, 509 U.S. at 591. This prong of the Daubert inquiry addresses the problem that arises when an expert’s methods, though impeccable, yield results that bear a dubious relationship to the questions on which he proposes to opine." (Emphasis added).
So, how to reconcile Saaman with Milward? I’m afraid it can’t be done.