Another attack on frequentist statistics
A news report in nature tells of yet another study concluding that Bayesian statistics are better than frequentist statistics. **Disclaimer: I don’t have time to read the actual scientific paper being reported, so the opinions that follow are about the Nature news report, not the original article**
John Ioannidis wrote a great paper a few years ago called “Why most published research findings are false”. However, Nature quotes his response to this new article, which to me is just too simple minded. Sure it could have been taken out of context, but in any case it is not a message I support,
“The family of Bayesian methods has been well developed over many decades now, but somehow we are stuck to using frequentist approaches,” says physician John Ioannidis of Stanford University in California, who studies the causes of non-reproducibility. “I hope this paper has better luck in changing the world.”
I will repeat my opinion on this kind of thing: (1) frequentist statistics are neither perfect nor terrible, (2) Bayesian statistics are neither perfect nor terrible, (3) it is possible to cheat with Bayesian statistics, (4) it is possible to cheat with frequentist statistics, and in conclusion, (5) the problem is not with this or that particular statistical paradigm, but rather with researchers really wanting to find results that are interesting…and therefore making interesting conclusions however they can (whether they is any truth to those conclusions or not).
Blaming a particular statistical paradigm is just a red herring. If we want science to be more reproducible, the scientific reward system needs to shift in favour of skepticism. This will have its downsides too, because if we don’t reward scientists making bold claims then science could be boring and may in fact fail to notice subtle but in-the-end interesting results. Of course the price of rewarding scientific boldness is many published results that are untrue.
The problem of how to encourage better scientific practice is at the intersection of the sociology of science and statistics (and methodology more generally). If you ignore one of these pieces (e.g. this recent Nature news report coming down on an entire statistical paradigm), then you will necessarily be oversimplifying the problem.