Science and Replication (or not)

An article today on FiveThirtyEight says it’s possible that the simple awarding of badges to those who follow certain procedures (openness of data, revealing methodology, etc), may have sparked an increase in these good practices. Or not, of course. The correlation is pretty clear. The causation is somewhat less so. It could be that all the publicity regarding replication has been more conscientious scientists choose more rigorous and open approaches.

What’s interesting to me is some of the popular fallout and commentary that I’ve heard on this. Completely unscientific quacks and nuts use the errors made by scientists as a cover for their own ridiculous claims, while ordinary people decide they can ignore scientific evidence because it might be wrong. If it might be wrong, why should I bother?

Projects such as the one awarding badges for certain practices can help with the public perception as well as the actual practice of science, and that will be a good thing.

What bothers me about the frequent response is that people who are actually much less careful and accountable than the scientific community use the misbehavior as some to both denigrate science as an enterprise and to open the door to completely untested ideas and theories.

Even though the scientific community has its problems—and I strongly commend those organizations working for greater transparency—your odds are still better working with material produced by a community that at least aims for sound methodology. Sure, some science may be wrong, but I’m willing to bet my life on my doctor’s views over those of someone who guesses and goes by their feelings.