Beyond Significance Testing: Statistics Reform in the Behavioral Sciences, Second Edition

Traditional education in statistics that emphasizes significance testing leaves researchers and students ill prepared to understand what their results really mean. Specifically, most researchers and students who do not have strong quantitative backgrounds have difficulty understanding outcomes of statistical tests.

As more and more people become aware of this problem, the emphasis on statistical significance in the reporting of results is declining. Increasingly, researchers are expected to describe the magnitudes and precisions of their findings and also their practical, theoretical, or clinical significance.

This accessibly written book reviews the controversy about significance testing, which has now crossed various disciplines as diverse as psychology, ecology, commerce, education, and biology, among others. It also introduces readers to alternative methods, especially effect size estimation (at both the group and case levels) and interval estimation (confidence intervals) in comparative studies. Basics of bootstrapping and Bayesian estimation are also considered. Research examples from substance abuse, education, learning, and other areas illustrate how to apply these methods.

A companion website promotes learning by providing chapter exercises and sample answers, downloadable raw data files for many research examples, and links to other useful websites.

New to this edition is coverage of robust statistical methods for parameter estimation, effect size estimation, and interval estimation. A new chapter covers the logic and illogic of significance testing. This edition also addresses recent developments such as the new requirements of some journals for the reporting of effect sizes.

Acknowledgments

Introduction

I. Fundamental Concepts

1. Changing Times
2. Sampling and Estimation
3. Logic and Illogic of Significance Testing
4. Cognitive Distortions in Significance Testing

II. Effect Size Estimation in Comparative Studies

5. Continuous Outcomes
6. Categorical Outcomes
7. Single-Factor Designs
8. Multifactor Designs

III. Alternatives to Significance Testing

9. Replication and Meta-Analysis
10. Bayesian Estimation and Best Practices Summary

References

Index

About the Author

Rex B. Kline, PhD, is a professor of psychology at Concordia University in Montréal, Canada. He has a doctorate in clinical psychology. His areas of research and writing include the psychometric evaluation of cognitive abilities, cognitive and scholastic assessment of children, structural equation modeling, the training of behavioral science researchers, and usability engineering in computer science. He has published six books and nine chapters in these areas.