Summary
background Researchers and reviewers often use the conventional p < 0.05 as threshold in statistical tests. In many cases, however, the interpretation of p-values is incorrect.
aim To explain where the 5% norm originates, identify the interpretation problems that often arise and suggest some alternatives.
method On the basis of recent literature we examine the meaning and origin of the p < 0.05 norm. We looked closely at entire articles and short reports in the Tijdschrift voor Psychiatrie, starting with the Jubilee issue of 2008, in order to find examples of methodological problems relating to the routine use of p-values.
results We found several examples of the problematic use of p-values; these included the testing of a priori unlikely, or even impossible null hypotheses, the reporting of small effects calculations based on erroneous assumptions, and incorrect interpretations of statistical parameters and p-values.
conclusion Research in psychiatry, like research in other disciplines, attaches too much weight to p-values. Guidelines for authors should advise authors to focus explicitly on effect sizes, confidence intervals and the scale on which the results are presented.