Experts Are Usually Wrong and Peer Review Makes Things Worse. Now What?
We've previously reported on the finding that when it comes to causal associations experts typically get it wrong. From the mass torts' perspective the problem seems to be that courts tend to confuse frequentist data with Bayesian degrees of belief - and thus did I get to see the late and truly very great trial lawyer John O'Quinn once convince a judge that any study showing an association that had a p-value less than .5 ought to be admissible and supportive of a verdict under Texas' "more likely than not" civil evidentiary standard. Armed with such a lax rule plaintiffs' experts ran wild. Heck, even today with a .05 threshold they are free to opine about causal associations that almost certainly do not exist.
The ability to manufacture health scares clearly serves a variety of political and economic interests but what to do about it? Doesn't peer review help? Not really. As long as the standard for statistical significance is low the odds are that even the most rigorous journals will publish more junk than science.
What about getting rid of peer review altogether? It's coming. In the future papers will be dipped in the acid bath of online skepticism and their ultimate worth will be determined by whether they survive the test. No longer will partisans, skirmishing behind the scenes, determine what gets published and what doesn't. See: "Scholars Test Web Alternative to Peer Review".
