Science Is Self-Correcting … Sort Of

August 16, 2011

One of the claimed advantages of scientific inquiry as a mode of acquiring knowledge is that it is a self-correcting enterprise.  For example, if someone claims to have discovered a process for cold fusion, that incorrect claim can be shown to be false or unwarranted by further research and scientific testing.  This rosy picture of the scientific enterprise suggests that scientific errors will be recognized and corrected, with false claims falling by the wayside during science’s inexorable march forward.

The reality is a bit more complicated than that, principally because we humans are the ones carrying out scientific inquiry, and therefore, scientific investigations are at risk of being tainted by all the faults human flesh is heir to, including stupidity, negligence, and deceit.

A recent article in the Wall Street Journal underscores both how prevalent errors are in scientific studies and how long it takes to uncover these errors.  Moreover, the number of retractions of scientific papers (retractions indicate an error has been detected) has been increasing rapidly this past decade.  Here are the numbers:  In the approximately 11,000 peer-reviewed scientific journals, the retraction rate has increased fifteen-fold from 2001.  There were only 22 retraction notices in 2001, but 339 last year and 210 through July 2011.

The time that it takes for science to “self-correct,” that is, for the retraction of a flawed study to appear, is also increasing in length.  In 2011, retractions were published, on average, within about five months of the original study.  The average now is over thirty-one months.

Three years may not seem like a long time to get it right, but if you have been taking medicine based on the original flawed study, that may be little consolation.  And, of course, in some cases it can take much longer.  The Journal article details the history of one flawed study that resulted in a significant change in the use of drugs to control hypertension.  Seven years later, after serious side-effects to some patients on the new regimen, the original study was shown to be flawed and possibly conducted unethically, without proper controls.

Perhaps most disturbingly, the Journal study reveals that about 26% of flawed studies are not the result of mere human error, but rather are the product of scientific fraud. In other words, a scientist has failed to follow proper procedures, for example, by cooking the data.  Why would a scientist engage in fraudulent conduct, especially if science is self-correcting?

Well, the motives could be many. First, the scientist may persuade herself that she is not doing anything wrong. She may be confident her theory is correct, so why does she need to follow all the necessary protocols? Her confidence may remain even if the data are not lining up quite the way they should. If she has the “right” answer, it’s not wrong to tweak the data is it? After all, once her theory is proven correct, no one is likely to notice or care (at least any time soon) that she has given the data a nudge or two.

Second, the scientist may hope that his improper methods are not discovered, or, if discovered, that they will be revealed only many years in the future. In the long run, we’re all dead, as many have observed.

And in the short run, one can obtain rewards, financial and otherwise, by making novel, important claims. Money and prestige are powerful motivations, for scientists as for anyone else. (Just make sure the results are not so extraordinary that they invite extraordinary scrutiny—as in the Korean human cloning scandal.)

Science is the best way to advance our understanding, no doubt. In part because of the success of science, ever bolder claims are made about its ability to provide us with important knowledge and guidance. Sam Harris, for example, has asserted that science can determine human values. Now is not the place to debate the validity of Harris’s thesis, but even if it were true, it’s one thing to know in the abstract what is morally right and wrong, and it’s quite another to adhere to those values in practice. Ultimately, science is only as dependable as the humans who apply it.