Mass Torts: State of the Art

Mass Torts: State of the Art

“It is indicative of a lack of understanding of the scientific method among many scientists”

Posted in Causality, Molecular Biology, Reason

For several years now we’ve been trying to spread the word to the legal community that a great many people who hold themselves out as scientists, including more than a few who’ve published papers in the most prestigious peer reviewed journals around, aren’t really doing science. They’re not coming up with hypotheses and testing them. Instead of avoiding that pitfall which humans are particularly prone to falling in, the one whereby we become so enamored of our clever hypotheses that we simultaneously become blind to any holes and hostile to those who dare point them out, too many scientists are fooled by the ability of statistical analysis to readily generate spurious associations that, with a little bit of post hoc narrative editing, look just like causal associations.

The combination of vast amounts of data quickly sliced and diced by powerful modern computers plus multiple statistical methods which are poorly understood but easy to use has led to the current crisis in bio-medical science whereby only a shockingly small fraction of ”scientific discoveries” turn out to be true. The essence of the problem is well put by the quote that appears in the subject line of this blog post. It’s from Donald Berry, a biostatistician at MD Anderson Cancer Center, and he made it during a discussion of the issue at last January’s meeting of the President’s Council of Advisors on Science and Technology. You can watch that portion of the conference dealing with irreproducible science here; it’ll take less than an hour of your time and is well worth it.

If you watch the webcast linked above you’ll hear concerned scientists explaining that a lot of other well-meaning scientists fail to comprehend the scientific method, are fooled by statistical tools they don’t understand, or both; and that more and better education is the answer. This idea, that with a little more of the right sort of education we’d get better science, assumes that nobody is trying to game the system. We’re to assume for example that: (1) no one is hatching his hypothesis after the computer has found the inevitable statistically significant associations that arise from looking at any bucket of data from multiple perspectives (if you doubt that finding something statistically significant in any random batch of numbers is easy then spend 60 seconds on An Exact Fishy Test); (2) no one is p-hacking his way to confirmatory evidence for his favored hypothesis by turning random noise into seeming proof; (3) no one consciously uses a test that is biased in favor of validating his method; and, (4) no one is exploiting the decision-making heuristics of peer reviewers and editors to sneak bad science into leading journals. If an article in this January’s The Cancer Letter is any indication we shouldn’t be too sure of such assumptions.

You need to read Duke Officials Silenced Med Student Who Reported Trouble in Anil Potti’s Lab for several reasons. First, it’s the story of a brave young man who risked his career by refusing to participate in and attempting to expose research practices that were shoddy at best and fraudulent at worst. Second, it’s about how an article published in Nature Medicine went from revolutionary to retracted. Third, it details how an institution dedicated to education was willfully blind to the rot that had set in at one of its most prominent laboratories even after the rot was pointed out. Fourth, it reminds us that bad science isn’t a victimless crime – that desperate cancer patients endure worthless and time-robbing clinical trials as a result of it. Finally, the article reminds us of the power of our adversarial legal system and the good it can do by bringing truth to light. Though the Institute of Medicine had investigated and Dr. Berry and others had pointed out the flaws in the since retracted article in the end everyone, perhaps out of a sense of collegiality, put the failings down to sloppy work and it looked like the worst thing Potti was guilty of was resume inflation. But then came the lawyers for the patients. They uncovered the emails and audio recordings showing, if intent can be inferred from conduct, that the data dredging, cherry-picking and non-test testing used to construct Potti’s revolutionary finding and to justify the clinical trials was done quite deliberately.

So enjoy the read, remember that bad science can be hard to spot, that provenance is no guarantee of good science, and maybe take a little pride in the fact that the tort system once again has helped to advance the cause of truth.

Chance Is Not A Thing: So you can’t lose it

Posted in Causality, Reason, The Law, Uncategorized

Conceptually the loss-of-a chance doctrine recently reaffirmed in Rash v. Providence Health & Services appears to make sense. The typical facts in such cases include (1) a usually fatal disease (e.g. certain cancers); (2) that was diagnosed later than was possible with proper care (or that a less effective treatment was used); and where (3) the limited chances of survival decline further with each successive stage of the disease’s progression. Not wanting to “provide a ‘blanket’ release from liability for doctors and hospitals any time there was less than a 50 percent chance of survival, regardless of how flagrant the negligence“, yet unable to come up with a sound reason why a plaintiff ought to be able to recover for an act or omission which probably did not cause the course of her disease to be altered, some courts made the erosion of the chance of survival the harm rather than the subsequent death. With that the causation dilemma seemed to disappear. Meanwhile, a mechanism for disincentivizing  (via the imposition of tort liability) the provision of anything less than optimal care, even to those unlikely to benefit from it, is created. One problem with the approach is that chance, especially in this setting, is not a thing that can be lost. Another comes from encouraging doctors to treat probability distributions instead of people.

Chance is a word imbued with powerful meanings. Often wrapped up in it are ideas about fate, destiny, fairness and even justice. Take the case of a simple coin flip that settles controversies from who kicks-off to who owns a $125,000 car. We may dispute the circumstances of the flip but never the outcome. Somehow, once in the air and spinning, fate, destiny, justice, karma or whatever hands down its unappealable judgment which is promptly revealed for all to plainly see. This idea of chance as a proxy for justice (or perhaps as a ward against injustice) is a particularly old one. Consider Jonah 1:7 :

Then the sailors said to each other, “Come, let us cast lots to find out who is responsible for this calamity.” They cast lots and the lot fell on Jonah.

Of course in the age of “Big Data” chance is supposed to be about the attempt to quantify our uncertainty. When we say “the odds are 50-50″ what we’re really saying is that we don’t have access to any information that would lead us to believe that one side is more likely to come up than the other. In this sense chance may be considered a measure of our ignorance of the mechanisms and/or variables that determine which side comes up.

Now the fact that it’s unappetizingly about uncertainty and ignorance wouldn’t be a good reason not to compensate someone who lost a chance like the one depicted in the coin toss scene from “No Country for Old Men”. There’s one chance you wouldn’t want to lose. Only in such a pure instance of chance can it become a thing you can lose; and that must be the concept of chance imagined by courts like the one that authored Rash. Unfortunately that’s not at all the sort of chance we’re talking about when we talk about the chance of surviving cancer.

Where do estimations of the chance of surviving cancer for five years come from? Obviously from other people and not the newly diagnosed. And did those other people all experience identical survival intervals? No. Even the graph of late stage pancreatic cancer patients has a long tail of the very lucky few. Consequently, any estimation of the central tendency of those other people, usually the median but sometimes the average survival time, homogenizes the experience of all the patients and produces a mathematically “typical” patient with an experience unlike any of the individual patients. Whereas the gas station cashier in the coin toss scene had the opportunity to save his life by choosing “heads”, to seize the opportunity presented by the graph of the survival experience of patients undergoing a new treatment the cancer patient would somehow have to be able to choose to be the “typical” patient; and that would mean being able to choose to have whatever currently unknown genetic and epigenetic makeup is responsible for the slightly improved “typical” survival time – which is impossible. You can’t buy that chance, and neither can you lose it.

The remaining argument for the loss-of-a-chance doctrine is that disincentivizing doctors from providing anything other than the treatment with the longest “typical” survival time at the earliest possible date would save some unidentifiable lives and so produce a benefit to society as a whole. This is where we wade into the widening controversy swirling around the use of statistics, despite (or rather because of) ignorance of underlying mechanisms and variables, to determine treatment. On one side are those who hold the view that “it is obsolete for the doctor to approach each patient strictly as an individual; medical decisions should be made on the basis of what is best for the population as a whole“. The idea here is that if earlier or a newer treatment has shifted the survival curve in the direction of longer survival in a subset of people with the disease then earlier or newer treatment across all people with the disease will surely save lives.

On the other side are those who point out that the medical journals (and law books) are littered with examples of treatments which demonstrated a pattern of better outcomes in a small population but which showed no benefit or worse outcomes once they were widely prescribed. That many researchers, doctors and pharmaceutical companies ”find some pattern in their data and they don’t even want to consider the possibility that it might not hold in the general population” is a well-known phenomenon.

As for our take on the controversy all we can say is that until the underlying mechanisms of cancer are elucidated inferring treatment from statistics is pretty much all we’ve got … but often that ain’t sayin’ much. Hopefully in the not too distant future physicians will look back on our current era and shake their heads at the thought of the primitives who settled upon cancer treatment options essentially by casting lots. That being said, to anchor liability on the claim that the slight positive shift in the probability distribution calculated for a small sample of likely terminal patients (in turn premised on the dubious assumption that patients can be thought of as so many balls in a quincunx machine getting chemotherapy an infinite number of times) will also be seen in a much larger sample of completely different likely terminal patients seems more than just a bit of a stretch.

Consider also the following: if the loss of the (imagined) chance is the harm, why don’t the people who lost the chance at the new treatment, but who responded to the old treatment anyway, have a claim? They lost a chance and that’s a harm after all. And what would the damages be for the harm? They’d be the same as they would be for the person who lost a chance at a treatment that probably wouldn’t have made a difference anyway, right? So why is it that some who are harmed have a claim while others who sufferer the identical harm do not? Because the loss-of-a-chance doctrine is incoherent.

In Rash the appellate court ultimately affirmed the dismissal of plaintiff’s claim because her expert couldn’t quantify the chance she had lost. That’s just another example of a court falling into the trap of believing that assigning numbers to things, even to things that are not things, makes them “scientific” so that, as here, damages may be “accurately” calculated. Yet it’s vital to the assumption that doctors are able to sell and patients are able to buy the “typical ” (mean or median) outcome of a treatment that actually yielded a wide range of outcomes, none of which were precisely “typical”. And the illusion of accuracy created by multiplying the quantified chance of the “typical” patient from small study by the value of someone else’s life to determine her damages is just that. But so it goes with the loss-of-a-chance doctrine.

However far science pushes back the shadows to reveal how the universe really works, chance retains its place as a somehow essential and inescapable aspect of our lives. Perhaps, as ably argued by a colleague recently when we were outlining this post, Garth Brooks nailed it when he sang “I’m glad I didn’t know, the way it all would end, the way it all would go. Our lives, are better left to chance, I could have missed the pain, but I’d have had to miss the dance”. Or maybe chance, once revealed as uncertainty, is actually the driving force behind mankind’s quest for truth. That’s my take. But whatever it is it’s not something you can buy at the doctor’s office.

About The Virus That Makes You Dumb

Posted in Causality, Microbiology, Reason

A colleague asked me yesterday what I thought about a story she’d seen in the media regarding a virus often found in algae. Supposedly it can impair human cognition. I told her that as a matter of fact I’d been working up a brief blog post on the topic because of its implications for mass tort litigation, gave her the short version and promised to finish the post and send it along once I’d finished working on a presentation. Now I have so here it is.

A couple of weeks ago "Chlorovirus ATCV-1 is Part of the Human Oropharyngeal Virome and is Associated With Changes in Cognitive Functions in Humans and Mice" was published in Proceedings of the National Academy of Sciences. It’s an excellent example of good science revealing just how strange our world really is and just how little we really know and how poor our guesses can be about the causes of human disease.

I’ll get to the strangeness and the upset applecart of prior beliefs shortly but first it’s important to point out that this is a classic case of scientific discovery. It’s a direct observation of something never seen before followed by an experiment designed to test the implication of its discovery. It is not a case of finding a correlation between one event and another that seemingly followed it and thereafter creating a model/explanation of how the observed association might be causal. That’s just generating a hypothesis. Nor is it the usual null hypothesis testing of small effects that drives most modern toxic tort litigation. That’s just pretending to be surprised when a system that has been slightly perturbed turns out to be slightly perturbed.

Like most modern discoveries this one depended on a new method of observation and curious scientists deploying that new method to peer about the world around them. The new method (methods really but I’ll cram them all into one category as they’re evolving rapidly and besides it shortens the post) involves rapidly (and cheaply) sequencing all of the genetic material in a sample, comparing it to libraries of microorganisms, human cancer cells (see e.g. The Cancer Cell Line Encyclopedia ), etc. and thereby "seeing" the biological diversity present in the sample. The name for it is metagenomics.

Gene sequencing isn’t new of course but what is new is the ability to sequence the unknown unknowns in a sample and to do so rapidly and (relatively) inexpensively. In the past you’d take a sample, culture whatever was present in a petri dish (multiple techniques here too but anyway …) and then sequence whatever was in each of the clumps of clones you’d cultured. The problem is that not every microbe is culturable. In some cases the right medium has yet to be found. In other cases the beastie is dead – an invisible casualty of an unseen battle between the immune system and an invader. Either way we were blind to much of what was going on.

In this study scientists took throat swabs from people participating in an investigation of cognitive ability (among other things) to see what they could see with their new method. Upon comparing the genetic data they found to the libraries available they were surprised to find a match with a virus that infects algae; one that had not previously been identified as being infectious in humans. They then looked to see if there were differences between those with and without the virus and found a marked decrease in cognitive ability among those infected. To test the hypothesis that springs so readily from such a correlation they then ran an experiment on lab animals. Sure enough, cognitive ability including memory declined in those mice inoculated with the virus. But that wasn’t the end of it. They next looked to see what genetic changes the virus had wrought and it turned out that it altered the expression of genes that have to do with cognition. Not quite proof but pretty compelling.

As the use of this new way of seeing, metagenomics, has expanded so have the number of discoveries. Have a look at Elevated Levels of Circulating DNA in Cardiovascular Disease Patients: Metagenomic Profiling of Microbiome in the Circulation and Pathogenic Microbes, the Microbiome, and Alzheimer’s Disease (AD) to see what may await.

And along with the discovery of new causes come ways to track it back up the causal chain (at least until a deep pocket is found). See for example Seeking the Source of Pseudomonas aeruginosa Infections In a Recently Opened Hospital: An Observational Study Using Whole-Genome Sequencing .

One question raised is what duty is owed to those of your workers and customers who might come in contact with what was thought to be a harmless green algae occasionally infected with a virus nobody suspected until now to have the ability to fiddle with your brain’s fine tuning. Another is whether you ought to have your cognitively impaired plaintiff who blames the landfill tested for Chlorovirus. Or how about whether there ought to be a REACh for food (since the companies will come to have, thanks to the use of metagenomic techniques to ensure food safety, vast libraries of the presumably harmless microbes and their ever evolving offspring that ride to market in and on their products)? And what about record-keeping requirements because even helpful bugs sometimes go bad? Is it foreseeable that an innocent bug shipped far from home and thrown in with a strange crowd might pick up a plasmid coded for virulence?

If scientific discoveries raise more questions than they answer then the intersection of the law and new discoveries ought to keep us busy.



Just When the USPSTF Recommends Type 2 Diabetes Screening for Everyone 45 and Over This Gets Published:

Posted in Rhetoric

Korea’s Thyroid-Cancer "Epidemic" – Screening and Overdiagnosis . The short version is as follows: the ability to screen for thyroid cancer led to an explosion (an almost unbelievable 15 fold increase) in its diagnosis. In fact, it’s now the most common cancer in South Korea. Nevertheless, the mortality rate hasn’t budged. That means the only thing screening managed to accomplish was thousands of needless biopsies and surgeries and their occasional complications, including death.

There’s an excellent Op-Ed piece by one of the paper’s authors in The New York Times that lays it all out but this line really caught my attention:

Too many epidemiologists concern themselves not with controlling infectious disease, but with hoping to find small health effects of environmental exposures – or worse, uncertain effects of minor genetic alterations. Perhaps they should instead monitor the more important risk to human health: epidemics of medical care.

Will this finding which clearly echoes the results of similar studies on the impact of mass screenings for prostate cancer and breast cancer and their respective diagnosis and mortality rates finally dampen the enthusiasm for screening? Not if the new screening recommendation for type 2 diabetes, and the public’s response to prior news about the impact of mass screenings, is any indication.

The U.S. Preventive Services Task Force (USPSTF) now recommends blood glucose screening for adults at increased risk of type 2 diabetes. Among those many risk factors are being aged 45 or older, BMI > 25 and having a close relative with type 2 diabetes. That’s not exactly everybody but it’s certainly a lot of bodies. So how many lives will be saved? Given that this is a Grade B recommendation and that the USPSTF’s review of the evidence led it to conclude that first-line therapy for diabetes prevention (aggressive lifestyle modifications) currently results in a lower incidence of diabetes, cardiovascular mortality and overall mortality you’d assume the answer would be "many". However, "the task force found inadequate direct evidence that measuring blood glucose leads to improvements in mortality or cardiovascular morbidity."

That doesn’t sound very promising. So why make such a recommendation? Who knows? Given the enormous political pressure recently brought to bear on the USPSTF (especially after its recommendation against screening mammography in women under 50) it would be easy to slide into cynicism and speculate about the potential motives of those (other than the undiscovered incipient diabetics) who stand to benefit from this new recommendation (remember: under ACA if the USPSTF recommends it, it gets paid for). Instead, since this is a mass torts blog, I’ll speculate about why the new thyroid-screening study notwithstanding, most folks will enthusiastically line up for another screening opportunity.

Despite widespread coverage and laudatory editorials in leading newspapers both left-leaning and right the evidence that mammography screening in those at negligible risk did more harm than good apparently had little effect. No Fall In Mammogram Rates After USPSTF Recommendations was the conclusion drawn from surveys of the attitudes of almost 28,000 women about screening. The same was true for elderly men and prostate cancer screening (though it must be noted that urologists and middle aged men have demonstrated significant reductions in rates of screening / being screened). So why do so many people go to the bother of having a diagnostic test that is sure to be an annoyance and almost just as sure to do no good? I propose it’s because of one of the greatest decision drivers of all and one effective plaintiff lawyers use to devastating effect. It’s fear of regret.

Think about the power of fear of regret in the context of the reluctance of parents exposed to stories about autism and vaccines to vaccinate their children. They’re weighing the chance of autism times its cost against the chance of irreversible brain damage from measles times its cost. Should they decide that the value of the former outweighs the value of the latter it can only be that, given the grossly unequal risks, they’ve added something quite heavy to that scale holding the price of their children not developing autism in order to bring it even with the costs their children not be brain damaged by measles. What could it be? I suspect that it’s fear of regret / fear of becoming blameworthy.

In looking back across some of the most effective jury arguments I’ve seen (as determined by damages subsequently awarded) it’s those taking the form of "If you don’t stop this company, one day you’ll learn of another death and that victim’s blood will be on your hands" that work best. It’s the argument that appeals not to sympathy but that rather instills the social fear of shame that does the most work. Be en garde.

Ignorance is $$$

Posted in Risk

Kip Viscusi has just published a very good paper about the GM ignition switch recall controversy. In it he argues that blockbuster jury awards in cases where corporations had made design and recall decisions based on cold economic calculations influenced GM’s decision not to do the same sort of risk analysis on its defective ignition switch problem. The result was a delayed recall and lost lives. Viscusi further argues against the admissibility of potentially inflammatory risk analysis evidence on the basis that when such analyses fairly (and highly) value human life they actually promote consumer welfare and so ought not be discouraged. The paper is "Pricing Lives for Corporate Risk Decisions" and can be found here.

GM isn’t the only company to have forsworn risk/benefit assessment. After a presentation to in-house counsel on recent trends among the courts regarding causation and risk, in which I discussed how not to do risk/benefit analyses, the GC of that energy company commented "we don’t do them (risk/benefit assessments) at all and we routinely counsel our business units not to engage in the practice." The sentiment is surely understandable given the usual media reaction to anything that sounds remotely like weighing profits and people on the same scales; and Viscusi’s paper lends further support to it with empirical evidence demonstrating that people, given identical facts about a defect and subsequent harm, tend to award significantly larger punitive damages against those defendants that examined the risks and benefits before they went to market than against those that either did nothing at all or that consciously chose not see how the numbers added up.

How then do companies that eschew risk/benefit analysis deal with risk? Sometimes by clinging to the belief that regulatory compliance renders their product or activity risk-free and sometimes by ignoring it altogether. Unfortunately, in either case it’s just whistling past the graveyard.

Remember that, generally speaking, a wrong is an action that was taken without proper regard for others and that caused some harm. Since every action entails risk (because every action entails change and every change entails risk, assuming the actor isn’t omniscient) any reasonable person who acts must either (a) be ignorant of the risk; (b) suspects a risk but chooses to leave things to the Fates; or (c) estimates that the benefits outweighs risks. Only the diligent then, those who have undertaken (c), have actually made an attempt at calculating the "proper regard for others" given the risk inevitably created.

Now of course it could be the case that a diligent actor is also a nefarious one who has grossly undervalued human life and/or has calculated that the chances his perfidy will be uncovered are slight, but such a case is the very one one for which punitive damages were designed. However, if the studies cited by Viscusi can be relied upon, then even the honest risk assessor (c) who highly values human life will get hammered harder by a jury than the ignorant (a) or the careless (b) manufacturer. Thus the perverse result: in an age when knowledge is ever more accessible, when questions about the likelihood and cost of harm posed by a particular design incorporated into millions of vehicles are readily calculable, and when a sound risk analysis can send a signal that saves lives we have a legal system that threatens enormous costs to any manufacturer who dares to look up the answers.

Viscusi’s solution is essentially to set the value of a human life (he suggests $9.1 million) at a level which, if used by a company in its risk/benefit analysis and if guided truly thereafter by the answer, would mean that the company was not putting its profits before people. Accordingly such an analysis could not be offered into evidence by the Plaintiff to demonstrate knowledge, callousness, etc. though it could be offered by the Defendant, at its option, to show proper regard. It’s a good idea and soundly reasoned. Read the whole thing as it’s well worth your time.

Yet Another Opinion in Which a Court Mistakes Hypothesis for Theory

Posted in Causality, Epidemiology, Reason, The Law

While some may imagine that scientific hypotheses are the product of highly educated people with brilliant minds drawing straightforward inferences from compelling evidence the fact remains that all scientific hypotheses are nothing more than guesses; and as every middle schooler taught the scientific method knows, even the best pedigreed hypotheses are usually false. On the other hand, sometimes it’s the hypothesis with the most dubious provenance that gets promoted to the status of scientific theory (i.e. one that has survived rigorous testing and is powerfully explanatory) as in the case of benzene’s structure:

I was sitting writing at my textbook but the work did not progress; my thoughts were elsewhere. I turned my chair to the fire and dozed. Again the atoms were gambolling before my eyes. This time the smaller groups kept modestly in the background. My mental eye, rendered more acute by the repeated visions of the kind, could now distinguish larger structures of manifold confirmation: long rows, sometimes more closely fitted together, all twining and twisting in snake like motion. But look! What was that? One of the snakes had seized hold of its own tail, and the form whirled mockingly before my eyes.

Because a hypothesis is nothing more than the assembly (by hard work or daydreaming) of a few bits of what is known/believed into a plausible narrative that explains some phenomenon (e.g. gastric lymphoma), because so little is known about the causes of a complex disease like gastric lymphoma such that the discovery of H. pylori suddenly and completely overturned prior views about its causes, and because we can’t know (or factor into our hypotheses) what we don’t know (you’ve heard of the human gut microbiome but what about the human gut virome?) hypotheses are nothing more than speculation. That’s why every epidemiological study you’ve ever read puts the burden of proof squarely on the hypothesis and resolves all doubt in favor of the "null hypothesis" (i.e. the hypothesized causal agent has no effect).

Unfortunately many courts either don’t understand the difference or refuse to distinguish between hypothesis and theory. A recent example is Walker v. Ford. In Walker plaintiff’s expert was allowed to opine on the basis of his hypothesis that asbestos is a cause of Hodgkin’s lymphoma and thereafter to deduce from another of his hypotheses (Hodgkin’s lymphoma is caused by either Epstein-Barr virus, smoking or asbestos) that plaintiff’s lymphoma must have been caused by asbestos as he hadn’t the virus and didn’t smoke. And it isn’t just another case of a court conflating hypothesis generation (guessing) with the scientific method (testing guesses) so that guesswork by a properly credentialed witness is turned into a "scientifically valid method" and Rule 702 can be deemed satisfied. It’s worse. Not only has the hypothesis that asbestos causes Hodgkin’s lymphoma never been verified, it has in fact been repeatedly tested and serially refuted. Furthermore, the most important observation that spawned the hypothesis in the first place (an increased risk of gastric lymphoma among a sample of asbestos workers) has never been reproduced (and will never be reproduced) because when the study was done nobody outside two researchers in Australia even knew H. pylori existed much less to look for it in gastric lymphoma patients – several years would elapse between its discovery and the determination that it is worldwide the leading cause of gastric lymphoma.

The general causation opinion of plaintiff’s expert rested on these studies:

1) Cancer Morbidity of Foundry Workers in Korea. A slight increased risk of stomach cancer and non-Hodgkins’s lymphoma was found among foundry workers exposed to a laundry list of things including asbestos. No exposure assessment was done for any substance and no increase in Hodgkin’s disease was reported. The mortality study of the workforce published this year isn’t any more persuasive – here’s the SMR table for malignant diseases: SMR table.

2) Extranodal marginal zone lymphoma of mucosa-associated lymphoid tissue type arising in the pleura with pleural fibrous plaques in a lathe worker. Guess what? Asbestos isn’t the only cause of pleural plaques and so I stopped reading this article when I got to "He had not been exposed to asbestos."

3) Asbestos exposure and lymphomas of the gastrointestinal tract and oral cavity. This is the study mentioned above that suffers fatally from the understandable ignorance of the confounder H. pylori though it also appears to have the multiple comparison problem as evidenced by the fact that subgroupings of lymphomas, here GI and oral, produced a higher risk than for lymphomas in general. Finally, being a case-control study, there was no estimation of exposure in any of the cases.

4) Does asbestos exposure cause non-Hodgkin’s lymphoma or related hematolymphoid cancers? A review of the epidemiologic literature. I didn’t get past the abstract which concludes that a review of the literature reveals "no increased risk of NHL (non-Hodgkin’s lymphoma) or other HL-CAs (hematolymphoid cancer) associated with asbestos exposure."

Not discussed in Walker but apparently the last nail in the asbestos-causes-lymphoma hypothesis’ coffin (and the last sign of any scientific interest in this apparently dead issue) occurred 10 years ago with the publication in the Annals of Epidemiology of Occupational asbestos exposure and the incidence of non-Hodgkin lymphoma of the gastrointestinal tract: an ecologic study. The study found "no support for the hypothesis that occupational asbestos exposure is related to the subsequent incidence of GINHL (gastrointestinal tract non-Hodgkin’s lymphoma).

These articles along with the expert’s belief that "as long as asbestos reaches an area, regardless of where it is, it can cause different types of cancer" and asbestos can make its way to the lymph nodes, were all he needed to opine that asbestos causes lymphoma including plaintiff’s Hodgkin’s lymphoma (because after all "a lymphoma is a lymphoma" save "for therapeutic purposes"). That’s too much nonsense to unpack in one blog post so I’ll just focus on the claim that wherever asbestos goes in the body it causes cancer. The Institute of Medicine was tasked with answering this very question – is there evidence for a causal relationship to asbestos for cancer of everything from the larynx to the rectum – and generally found that what was in the literature was suggestive but insufficient to reasonably conclude that there is a causal link. See: Asbestos: Selected Cancers.

To save plaintiff’s expert and his hypothesis the appellate court held that it doesn’t matter if an expert’s conclusions are correct. All that matters is that the method whereby he reaches his opinion is reliable, and plaintiff’s expert’s method, guessing about the cause of Hodgkin’s lymphoma by creating a narrative about the causation of Hodgkin’s lymphoma from a few studies (that didn’t actually study Hodgkin’s lymphoma) counts as a reliable one. But who, other than the hopelessly ironic, would label as "reliable" a method (i.e. the guessing that constitutes a scientific hypothesis) of causal determination the product of which is usually incorrect? Recall that not only are most scientific hypotheses false but that even most of those with a statistically significant chance of being true are probably false.

Only scientific theories get the Seal of Reliability, which is to say they make predictions on which you can rely. And they gain that status only by being put to the test, and passing; and by passing I mean that the predictions they make actually come to pass. So what prediction would follow from "asbestos exposure causes Hodgkin’s disease"? Wouldn’t it be "people exposed to asbestos are more likely to get Hodgkin’s disease than those who aren’t"? And what follows from the fact that no study of asbestos-exposed workers has shown an increased risk of Hodgkin’s lymphoma? That the claim "asbestos causes Hodgkin’s disease" isn’t reliable.

So if hypotheses are unreliable in general because by definition they have not been tested, and if the specific hypothesis "asbestos causes Hodgkin’s lymphoma" is unreliable because it has been tested and failed to predict the future it entails, in what sense is the opinion of Walker’s expert "reliable"? Let me know if you figure it out.


Posted in Causality, Microbiology, Risk, Toxicology

Jumping the Snark: Erionite in Mexican Town Tied to High Rate of Mesothelioma (or, how "Sir, have you ever been to Turkey?" became too cute by half)

Plaintiffs’ Experts Were Really, Really Wrong About the Mechanism Underlying MDS and AML

Chromium VI is Weakly Associated With Stomach Cancer

To Reduce the Spread of Pathogens in Common Areas Consider Shark Skin

Telomere Length is Like Life: You Gotta Take the Malignant Melanoma Risk With the Lifespan Boost

A Plaintiff Win and a Very Good Daubert Opinion from the U.S. Ninth Court of Appeals

Posted in Reason

… we have a universally recognized Supreme Court, to which all disputes are taken eventually, and from whose verdict there is no appeal. I refer, of course, to direct experimental observation of the facts.

 E. T. Jaynes, physicist, in Foundations of  Probability Theory, Statistical Inference, and Statistical Theories of Science (1976)

Recently we’ve been grousing about Daubert-invoking opinions that are actually derived from the strange belief that reliable scientific knowledge does not depend upon the existence of supporting observable facts. Today however we’re applauding City of Pomona v. SQM North America Corporation; an opinion that gets Daubert and sound science right because it gets the scientific method right.

At issue was the trial court’s order excluding an expert witness for the City of Pomona in a groundwater perchlorate contamination case. The expert, Dr. Neil Sturchio, was prepared to testify that the perchlorate detected in the city’s groundwater had most likely originated in the Atacama Desert of Chile. Such evidence, if admissible, would implicate defendant SQM as it had imported for sale to California’s agricultural industry many thousands of metric tons of sodium nitrate, an inorganic nitrogen fertilizer, from Chile’s Atacama Desert – the sodium nitrate from which contains on average about 0.1% perchlorate.

It’s a big world though and there are plenty of potential sources of perchlorate in groundwater. Perchlorate occurs naturally and has been found in relative abundance in some arid regions of the American Southwest. It’s also synthesized industrially for use in solid fuel rocket propulsion systems (manufacturers of which were located in and around Pomona) and fireworks so that defense contractors, the Cal Poly Rocketry Club and 4th of July celebrations all fall under suspicion. How then did Dr. Sturchio determine that the perchlorate in Pomona’s groundwater came from a desert almost 6,000 miles away? Well, he didn’t conduct an experiment in his mind (a la McMunn and Harris) involving unobservable facts (a la Messick) in order to generate an untestable conclusion (a la Milward). Instead he followed the scientific method.

Perchlorate consists of one chlorine atom and four oxygen atoms. Since both chlorine and oxygen atoms come in different varieties (called isotopes) the question arose as to whether perchlorate also comes in different varieties defined by the ratio (or distribution) of the various isotopes making up its component parts. If it does, and if those varieties vary according to where they’re from or how they’re made, then something akin to a fingerprint or signature could be generated by analyzing the relative distribution of isotopes in a perchlorate sample. Finding a match in a database of known perchlorate signatures would thereafter yield a suspect and so satisfy plaintiff’s evidentiary burden of production.

According to Dr. Sturchio there is a method (it’s a subtype of mass spectroscopy known as isotope-ratio, or IRMS) for assessing the ratio of isotopes (or variety) of a particular perchlorate sample and there is a list, albeit a short one, of known perchlorate samples and their IRMS signatures. Most importantly there’s a detailed report available that sets out the theory, its rationale, the prediction it makes, the method by which it was tested and the results obtained. Using that method and comparing the results from prior tests on other samples showed, according to Dr. Sturchio, that the perchlorate in the Pomona, CA groundwater had a fingerprint, a signature if you will, just like that of Chilean perchlorate; and not at all like that of either man-made or indigenous perchlorate. Thus his opinion on its origin.

The district court found Dr. Sturchio’s opinion to be unreliable for three reasons. The first was that no government agency had yet certified the method he employed and that it was still being revised. The second was that the particular procedure he had followed in this instance had not been tested and could not be retested. The third was that the database of known perchlorate varieties and their signatures was too small.

Because the theory that perchlorate comes in different varieties has been tested and corroborated by at least one study, and because those varieties can per the theory be distinguished by use of a widely available method, the Ninth quite rightly held that the criteria at the heart of Daubert, testability and reliability, had been satisfied. Defendant was free to test the theory as well as the method and to refute the former and/or explain why the latter was either inappropriate for the task or so inaccurate as to be unreliable, Without evidence of its own to refute plaintiff’s corroborated theory or the accuracy of the method used to test it Dr. Sturchio’s opinions could not be excluded. In such a case, the court held, arguments aimed at the potential for holes in the theory or error in the methodology go to the weight and not the admissibility of the proffered evidence.

The court did a nice job (with one exception to be discussed below) in setting out its reasoning and so rather than go on about it we’ll just refer you to the opinion. Instead we’ll make one comment and then try to explain why the court’s argument against the need for certainty isn’t the usual straw man argument but is rather an understandably awkward attempt to address the problem known as "the glory of science and the scandal of philosophy".  

Our comment goes to the defendant’s argument that the reliability of a scientific method depends upon final government certification of the particular technique. We file it under "Be careful what you wish for" and remind our readers that the power to define what is and what is not a telescope determines what can and cannot be seen through one.

Now for the hard part. Defendant’s argument that the database of perchlorate signatures was too small and the court’s response that the law doesn’t require certainty is just a version of a very old argument that goes like this:

Scientist: If all swans are white then every swan that is seen shall be white. Since all swans ever seen have been white, all swans must be white. It is thus my opinion that if the color of a particular swan is at issue then it will have been white.

Inquisitor: I shall demonstrate that you have fallen into the fallacy known as Affirming the Consequent; which is recognized by all learned men and women to constitute a notorious error of logic. Have you seen, or seen a record of, every swan that lives or has ever lived?

Scientist: I have not.

Inquisitor: So there are swans that have been that have never been seen and swans now living that remain undiscovered and yet you have no knowledge of their color?

Scientist: it is undeniable; and I would add that there are swans yet to be and I am of the opinion that they too will be white.

Inquisitor: Then you further admit that there is an entire category of things, that of swans yet to be, about the color of which you are willing to opine despite having not a single observation of any member of that category?

Scientist: I do. It is after all prediction on which a scientist stakes her reputation.

Inquisitor: A risky gamble, is it not? For despite ten thousand observations of white swans the sighting of a single black one would do what to your theory that all swans are white?

Scientist: It would refute it, utterly.

Inquisitor: How then can your theory be anything more than the most tenuous sort of idle speculation - built out of nothing more than anecdotes, a collection of sightings of some unknown portion of all swans, and at risk of collapse at any moment?

Scientist: Because all those confirmatory sightings with none to the contrary make it at least very probably true.

Inquisitor: Alas, I had hoped you had something more and might save me from my skepticism. Oh well. Do you agree that probability is a measure of uncertainty?

Scientist: Yes, it plays that role in logic.

Inquisitor: And what is meant by uncertainty?

Scientist: To take the case of a coin toss, for example, while I cannot tell you whether when flipped it will come up heads or tails I can tell you the expected frequency of each occurrence; or, given past experience flipping the same coin, I can tell you to what degree I believe the coin will come up say heads. Any probability calculated other than 100% is then a measure of uncertainty.

Inquisitor: But  to make such calculations you must know how many sides the coin has or how many cards are in the deck?

Scientist: True.

Inquisitor: How many swans have there ever been?

Scientist: I do not know.

Inquisitor: And how many swans are yet to be?

Scientist: I do not know that either.

Inquisitor: So you cannot say whether your record of white swan observations constitutes a large, small or merely infinitesimal fraction of all the swans that have ever been or will ever be?

Scientist: I cannot

Inquisitor: And so when you use the word probability you use it in neither the mathematical nor the logical sense, since you have insufficient information to even estimate the probability that your opinion is correct?

Scientist: That is true.

Inquisitor: So now, at last, do you admit that you really do not know?

Scientist (revealed as Sir A. B Hill): Who knows, asked Robert Browning, but the world may end tonight? True, but on available evidence most of us make ready to commute at 8:30 the next day.

In City of Pomona plaintiff’s expert had real evidence – facts that are directly observable and subject to testing – and not just a hunch. Those facts and the method by which they had been gathered had been tested by others, and confirmed. And defendant was free to find a black swan. That, in our view, is good enough.

Three Straw Men on a Witch Hunt

Posted in Reason

He who is accused of sorcery should never be acquitted, unless the malice of the prosecutor be clearer than the sun; for it is so difficult to bring full proof of this secret crime, that out of a million witches not one would be convicted if the usual course were followed!

- 17th century French legal authority

While looking for some references to help make sense of the (tortured?) reasoning responsible for Messick v. Novartis Pharmaceuticals,Corp. (a recent U.S. Ninth Circuit opinion arising out of a California intravenous bisphosphonate/osteonecrosis-of-the-jaw (ONJ) case) we came across the quote above in the very enjoyable Does Your Model Weigh the Same as a Duck? Though the aim of that paper is to expose two "particularly pernicious" fallacies of logic infecting drug research methodology, the point it makes when referring to the pre-Enlightenment era’s view of the appropriate evidentiary burden in the trial of witches applies equally to modern courts that lower standards of proof in toxic tort cases lest, they fear, all present day witches (i.e. chemicals/pharmaceuticals) go unburnt.

A strong suspicion that the usual course, i.e. skeptical gatekeeping, won’t be followed in Messick arises early on when the court chooses to demonstrate the strength of its argument (that the trial court erred when it found the opinions of plaintiff’s causation expert to be irrelevant and unreliable) by fighting not one but three causal straw men. As usual there’s Certainty, which advances the quixotic claim that plaintiff needs to prove causation essentially by deduction ( failing to recall Hume, D., A Treatise of Human Nature: "all knowledge degenerates into probability"). Then there’s How, which stands for an argument nobody makes – e.g. we can’t reasonably infer that aspirin reduces the risk of heart disease until it is proven how it does so. Last there’s Sole, which defends the defenseless argument that a putative cause must also be the sole (i.e. only and sufficient) cause of plaintiff’s injury. Unsurprisingly, each straw man is dispatched in a paragraph or less.

What is surprising is the length to which the court goes to save the plaintiff from her own expert, Dr. Richard Jackson. Jackson admitted that the fact that bisphosphonates are a cause of ONJ doesn’t mean that it was the cause of her ONJ. He even admitted that the plaintiff had multiple risk factors for ONJ and that he could not determine "which of those particular risk factors is causing [the ONJ]." You would think that such equivocal testimony would put an end to plaintiff’s quest to prove causation and that’s exactly what the District Court below had held; but the Ninth Circuit thought otherwise.

The appellate court held that while plaintiff’s causation expert "never explicitly stated that Messick’s bisphosphonate use caused her [ONJ]", Dr. Jackson had analogized plaintiff’s use of bisphosphonates to "the oxygen necessary to start a fire." Also, he had said that "[bisphosphonate use] was at least a substantial factor in her development of [ONJ]." Finally he was prepared to opine, based on his "extensive clinical experience", that "a patient without cancer or exposure to radiation in the mouth area would not develop ONJ lasting for years (as had plaintiff) without IV bisphosphonate treatments". Somehow, that’s enough for a plaintiff to get to the jury on causation.

The problem with the assertion that bisphosphonates are to ONJ as oxygen is to fire is that a quick PubMed search reveals numerous cases of ONJ in cancer patients decades before bisphosphonates were ever marketed to them. ONJ has been attributed to bacteria, dental work, radiation and cancer all by itself. Perhaps, given that Dr. Jackson diagnosed plaintiff with bisphosphonate-related ONJ (or BRONJ), he’s really saying: "plaintiff has BRONJ; therefore she has ONJ related to bisphosphonates". But that would just be begging the question and presumably not persuasive to the court. Either way, how an argument that’s either demonstrably false or a logical fallacy can support plaintiff’s causal claim escapes us.

Next, what should we make of the court’s reliance on Dr. Jackson’s "it’s at least a substantial factor" opinion? Apparently what the court is saying is that while (1) there are multiple causes of ONJ including bisphosphonates; and (2) plaintiff had cancer, and perhaps other risk factors, known to cause ONJ; even though (3) her expert can’t say which one did it; because (4) he’s prepared to testify "that Messick’s bisphosphonate use was a substantial factor"; (5) such testimony satisfies California’s substantial factor standard and is admissible. However, the only way that (5) follows from (1 – 4) is if proof of "but for", or counterfactual, causation is not an element of California’s substantial factor causation test and "maybe" causes are good enough. Yet California’s substantial factor standard actually "subsumes the ‘but for’ test". We’re again left scratching our heads.

The final (and apparently to the court most compelling) causation argument was that Dr. Jackson, on the basis of things seen only by himself (i.e. his clinical experience), had ruled out leading alternate causes and so thereby reliably ruled in bisphosphonates. This is of course just ipse dixit making an appearance in its de rigueur guise as "differential diagnosis" – the court admitting as much when it writes "[m]edicine partakes of art as well as science …" while pretending not to notice the impact of the evidence-based medicine revolution. We’ve taken the position in prior posts that believing differential diagnosis (a/k/a differential etiology a/k/a inference to the best explanation) to be akin to the scientific method and that it produces the sort of reliable scientific knowledge contemplated by Rule 702 is simply the sort of pre-scientific thinking common among those prone to being mesmerized by credentials and jargon. Instead of rehashing those arguments consider the case of Dr. Franz Mesmer and what is revealed when the scientific method is applied to the beliefs of doctors drawn from their clinical experience.

After having seen many patients Dr. Mesmer came up with a hunch about how the body worked and how good health could be restored to the sick. His hypothesis was called animal magnetism and it entailed that an invisible force ran through channels in the body which, when properly directed, could effect all manner of cures. Redirecting that force via mesmerization became wildly popular and Dr. Mesmer became quite famous. In what would become the first recorded "blind" experiment clinicians who practiced mesmerism – the art of redirecting the invisible forces to where they were needed – proved unable, when they did not know what it was they were mesmerizing, to distinguish a flask of water from a living thing and neither could they produce any cures. On the commission overseeing the experiment in 1784 was none other than one of the leading lights of the American Enlightenment and rebel against authority – Benjamin Franklin.

Two hundred and ten years later doctors were still seeing in their patients what their hypotheses predicted rather than what was actually occurring. A classic research paper demonstrating the phenomenon is The impact of blinding on the results of a randomized, placebo-controlled multiple sclerosis clinical trial. Investigators assessing the efficacy of a new treatment for multiple sclerosis (MS) in a flash of brilliance decided to "blind" some of the neurologists who would be clinically assessing patients undergoing one of three treatments while letting the rest of the neurologists involved in the effort know whether a particular patient was getting the new treatment, the old treatment or the sham treatment. While the blinded neurologists and even the patients who had correctly guessed their treatment assignments (a check for the placebo effect) saw no improvement over the old treatment, the unblinded neurologists not only saw a significant positive effect that wasn’t there but they continued to see it for two years. Is there some workaround; a way to test after the fact for the distortion of the lens through which a clinician in the know observes his or her patients? You could try but it would appear to be a mug’s game and furthermore, by the very nature of the bias produced (unblinded clinicians blind to the very existence of their own bias) beyond the ability of cross examination to uncover.

A clinician’s art and a differential diagnosis derived from that art saved the day in Messick. Along the way to deciding that objective, verifiable evidence is not required to prove causation in such cases the court listed her sister circuits said to be of like mind and in the first footnote added that the Fifth Circuit was now alone in not having similarly lowered the gates. How the Fifth Circuit feels about being Daubert‘s last redoubt is unknown to us but we’re pretty sure that a plaintiff would win on causation in a bisphosphonate-ONJ case before that court. That’s because there are five years worth of objective and verifiable (and verified) evidence that (a) ONJ incidence in bisphosphonate-treated cancer patients is drastically and consistently increased; and (b) the likelihood that ONJ in a bisphosphonate-treated cancer patient was due to the treatment is slightly over 98%. See: 2014 AAOMS Position Paper on Medication-Related Osteonecrosis of the Jaw.

We know why plaintiffs’ counsel don’t want courts to embrace the sort of causal reasoning that would make a case like Messick easy for both general and specific causation. It’s because the day a court holds that "a probability estimate of 98% obviously passes the ‘more likely than not’ test" is the prelude to doomsday in low dose asbestos/benzene/etc litigation when that same court holds "a probability estimate of 2% obviously does not’". What we can’t understand is why so many courts refuse to enforce the test by demanding something more than the musings of experts. Witches or bewitchment are our two working hypotheses.



Posted in Microbiology, Molecular Biology

Some months ago we decided to put Discretizations on hiatus while we tried to figure out what to do about the tsunami of scientific papers washing up on PubMed (which was already piled deep in the flotsam and jetsam of publish or perish) now that China, India, Korea, etc. are getting in on the fun. On the one hand we’re sorely tempted to take advantage of a situation that presents us daily with awesome posting opportunities like: Hay fever causes testicular cancer! On the other hand we know that most statistically significant findings are in fact false - in no small part because "it is a habit of mankind to entrust to careless hope what they long for, and to use sovereign reason to thrust aside what they do not desire". So, rather than being a part of the problem by propagating noise and not signal we’ve decided to limit our Discretizations to papers that 1) report an actual observation and not just the spectre of one drawn from data dredging and statistical analysis; 2) reflect an effort to reproduce a previously reported finding (e.g. testing a hypothesis previously drawn from statistical inference but in a different population); or, 3) extend (marginally) some previously established finding. Here goes:

Your word for the day is "immunobiotics": good bacteria that can aid in the production of certain blood cells

Obesity is a risk factor for lumbar radicular pain and sciatica – and it may impair healing mechanisms

Poor folks still have (on average) poor ways (which explains a large portion of the disparity in life expectancy between those in the bottom quartile of socioeconomic status and everyone else) 

Hospital staff and visitors can spread nosocomial pathogens to nearby businesses

How does metformin help patients with diabetes? Maybe by making the gut a happier place for A. muciniphila

No evidence found to support the hypothesis that parental smoking (and the benzene exposure that goes with it) during pregnancy increases a child’s risk of developing acute lymphoblastic leukemia (ALL)

Does the debate about segregating cystic fibrosis patients by lung/bronchial microbiome, going so far as to prevent some/most/all-but-one CF sufferers from attending indoor CF Foundation events, presage future debates as more and more diseases are found to have bacterial components? Here are the pros and cons of the CF debate.