Questionable research practices, low statistical power, publication bias, and the current replicability crisis

In recent years, there is a growing understanding that empirical sciences are in a state of crisis: the replicability crises. In diverse areas, such as cancer research, neuroscience or social psychology, systematic attempts at replicating pivotal research findings fail in the majority of cases. In this talk, I will present the scope of this problem and then attempt to map its causes. These include data fabrication, as well as honest errors in data analysis and reporting; “Questionable Research Practices” (QRP, techniques that allow to reach the Holy Grail of p < .05), especially HARKing (Hypothesizing After Results are Known); and thoughtless application of the dominant Null Hypothesis Statistical Testing (NHST) approach. Then I will review several practices intended to improve the quality of reported research that currently gain in popularity: using larger samples; pre-registration of research hypotheses, methods, and analyses; formal statements assuring the full report of all relevant information in a publication; detailed method description in on-line supplements; public data sharing; and having statisticians involved in all the stages of research planning and implementation. The change in existing practices is slow, but it is confidently spreading to more and more scientific disciplines.