In light of the NYTimes' "A Sharp Rise in Retractions Prompts Calls for Reform" we thought it would be a good time to revisit some of our objections to "Reference Manual on Scientific Evidence (Third Edition)"; particularly the second chapter, "How Science Works". Here goes.
Avoiding any pretense of humility the Reference Manual dismisses as woefully naive and inadequate those claims about the essence of the scientific endeavor that were ingrained in us in school. Sir Francis Bacon's scientific method? "[E]ven ... in his own time there were those who knew better." The idea that scientists ought to be unbiased observers of nature? What is seen from the shoulders upon which observers stand in order to see a bit further depends as much on the giant as on the observer; meaning somehow "Bacon has been left behind". In the chapter's section on myths and facts about science the Reference Manual says "Myth: Scientists are people of uncompromising honesty and integrity. Fact: They would have to be if Bacon were right about how science works, but he was not." Talk about cynical.
Then there's the section on Sir Karl Popper, unaffectionately known among some academics as "the man who murdered Karl Marx and Sigmund Freud". Seeing no use for hypotheses that were infinitely explanatory yet unable to accurately predict anything, Popper accepted Hume's problem of induction but found in those theories that made claims about what would follow if they were true a way to distinguish science from pseudoscience. Can the hypothesis be put to a test that it would fail were it not true? This is the criterion of falsifiability which found its way into Daubert.
The Reference Manual (like so many attacks on Popper since Daubert came out) starts by body-slamming pro wrestling-style a straw man: the claim that Popper believed a good scientist should think mightily, perceive a pattern, find a sound explanation and then spend the rest of her days attempting to prove herself a fool; meanwhile taking no sensible action that would follow from her hypothesis. To the contrary, Popper clearly opined that we ought to act on the best available evidence while keeping an open mind.
The Reference Manual then points out that a falsified theory might actually be an indictment of the giant upon whose shoulders the scientist stands rather than her reasoning. Sure. It might be true that everything we thought we knew about something is wrong (and revolutions do happen - think e.g. the human microbiome) but such a claim would be extraordinary and demand extraordinary proof. Barring such proof we're pretty safe concluding that a claim requiring we throw out everything we know is likely false. Unsurprisingly the Reference Manual, operating on the view that objectivity is an illusion, that you can never prove anything is false and that you can never prove anything is true ("the apparent asymmetry between falsification and verification that lies at the heart of Popper's theory thus vanishes") and thus without any track to follow, quickly careens into post-modernism.
Suddenly "science" is about context. Suddenly "science" is no longer about a quest for truth. "It takes a great deal of hard work to come up with a new theory that is consistent with nearly everything that is known in any area of science. Popper’s notion that the scientist’s duty is then to attack that theory at its most vulnerable point is fundamentally inconsistent with human nature." Science is thus about self-interest and power:
Myth: Scientists must have open minds, being ready to discard old ideas in favor of new ones.
Fact: Because science is an adversary process in which each idea deserves the most vigorous possible defense, it is useful for the successful progress of science that scientists tenaciously hang on to their own ideas, even in the face of contrary evidence (and they do, they do).
Of Thomas Kuhn the Reference Manual shrugs and says that all that business about paradigms collapsing to be replaced by new ones is similarly unevolved thinking. Instead "science does not, as Kuhn seemed to think, periodically self-destruct and need to start over again, but it does undergo startling changes of perspective that lead to new and, invariably, better ways of understanding the world. Thus, science does not proceed smoothly and incrementally, but it is one of the few areas of human endeavor that is truly progressive." One imagines the author spray painting "Ptolemy Lives!" on a subway wall.
Finally, even poor Galileo gets thrown in the ash bin of scientific history. Remember "the authority of thousands is not worth the humble reasoning of one single person"? False (or so says the Reference Manual). "[A]uthority is of fundamental importance to science... The triumph of reason over authority is just one of the many myths about science ..." Ugh.
So all the great thinkers were wrong. Objectivity is out. Testability is out. Keeping an open mind is out. Skepticism is right out. The appeal to authority is not a logical fallacy but fundamental to science. And supposedly it all adds up to making 21st century science conducted under such an understanding the best ever since, according to the Reference Manual: "There is no doubt at all that twentieth century science is better than nineteenth century science, and we can be absolutely confident that what will come along in the twenty-first century will be better still." So how's that working out for us so far in the 21st century?
Not so well. From the NYTimes article linked above it appears that bad science, unreproducible science and downright fraudulent science are all way up. We find ourselves in a "dysfunctional scientific climate." The problem? "You can't afford to fail, to have your hypothesis disproven." "It's a much more insidious thing that you feel compelled to put the best face on everything." That's it then. The conduct the Reference Manual calls science, hatching and clinging tenaciously and unquestioningly to a pet theory, even in the face of falsifying evidence, in hopes of becoming an authority in order to get more money just to repeat the cycle all over again, has led to a crisis in science. And the solution? Happily Dr. Casadevall, on the committee that oversees the writing of the Reference Manual, preaches giving graduate students a better understanding of what science ought to be - "the science of how you know what you know". Dollars to donuts the sermon will be long on critical thinking and real short on appeals to authority.
Science gained its prestige and respect not only from its ability to predict (and thus to allow us to make better choices) but also from its promise to respect knowledge, however humble she who reasoned it out. And despite whatever rot and corruption has crept into it in the last 20 years, the science that Daubert embraced still commands enormous respect. It's a science undaunted by authority, unimpressed by mere credentials and unafraid to dip any belief we hold dear in its acid bath of skepticism. And, especially in the biosciences, it's on the verge of sparking changes on par with, or more likely surpassing, those that followed the industrial revolution.
