I discuss problems with the use of null hypothesis significance testing, as it is particularly applied to safety research such as that in aviation psychology. Such problems are manifest in the inherent bias of traditional statistics to avoid type 1 statistical errors, and hence to discourage findings of safety improving effects as significant, when low powered experimental designs are required by necessary constraints. In contrast, I offer several approaches or remedies. Researchers should think about the decisions made by consumers of their research, based on the costs and values of those decisions; they should form alternative hypotheses, use smart planned comparisons where possible, and present data on the size of effects that do not meet conventional .05 levels of significance. Meta analyses are also encouraged.
Wickens, C. D.
(2015). Statistical Errors in Aviation Psychology: Commonsense Statistics in Aviation Safety Research. 18th International Symposium on Aviation Psychology, 348-353.