The Differential-Diagnosis "Methodology"
The case of Pluck v. BP Oil Pipeline Company, decided last week by the U.S. Sixth Circuit, turned on whether the opinions of plaintiffs' expert were properly excluded as unreliable and on whether his attempt to salvage them, by subsequently filing a supplemental report stating that upon using the court-approved "differential-diagnosis methodology" the identical evidence-free opinions had (not surprisingly) been reached, was timely. The court answered "yes" and "no", respectively.
Along the way to reaching its decision the court restated its view of the soundness of the differential-diagnosis methodology in deciding the cause of a individual's illness. Lots of courts have been saying the same thing of late. But all they're really saying is that using a decision tree to make a decision is OK. That's like saying that using a digital calculator to calculate the length of the hypotenuse of a right triangle is OK. It doesn't, however, say anything about about whether the data used was accurate nor even about whether the determinative quantities had been measured in the first place.
When we think about differential diagnosis we ought to be thinking of something like this excellent example from Baylor College of Medicine's Radiology Club. Working from a variety of well established causes of a liver mass and precisely measured tests for the presence of each the physicians were able to rule in cholangiocarcinoma while methodically ruling out a variety of other potential causes. We'll save for another day the question of whether or not such a methodology was ever intended to, or is any more capable than cast dice or chicken entrails of, identifying heretofore unrecognized causes of a particular illness a la Milward.
Instead, here's what can happen when a court is satisfied with the utterance of the magic phrase "I used the differential-diagnosis method" and decides it's up to the jury to determine whether the expert's rulings-in and rulings-out are sound. We'll change the facts to those of a well-known example to protect the innocent and to keep us out of trouble with a certain court.
Consider the following, modified from "Judgment Under Uncertainty: Heuristics and Biases". The known, possible causes of plaintiff's illness are either "genes" or "tetramethyl-death". Bad genes account for 85% of all cases while "tetramethyl-death" is only rarely indicted, accounting for just 15% of the cases. Plaintiff's expert says that plaintiff was exposed to "tetramethyl-death" and didn't have the bad genes. The court believes (perhaps because plaintiff's expert has made the mistake of going out on a Bayesian limb and giving an estimate of his faith in his estimation, hint, hint) that 80% of the time plaintiff's expert is able to accurately distinguish between cases caused by "tetramethyl-death" and those caused by bad genes. What then are the odds that plaintiff's illness was indeed caused by the chemical exposure rather than his genes?
Significantly less than 50%.
So the question becomes, essentially, should a verdict finding that one and one sums to three be upheld assuming the parties had a full and fair opportunity to cross examine the purveyor of such nonsense? A surprising number of jurists say "yes". That's what happens when you don't examine the alleged support for each branch of the decision tree; and that's what happens when you don't "get" percentages.
