The State of Our Science: Executive Board Perspectives
By SPSP Board of Directors
Science advances largely by correcting errors, and scientific progress involves learning from mistakes. By eliminating errors in methods and theories, we provide a stronger evidentiary basis for science that allows us to better describe events, predict what will happen, and solve problems.
We are currently seeing this process in action through the controversies about scientific practice and the replicability of social and personality psychology findings. This post is designed to provide our view of these events, along with links to some of the resources where this debate and discussion is ongoing. We also invite readers to provide their own commentary by posting on the Best Practices section of the SPSP Blog, Character and Context.
If you are just getting caught up on the diversity of viewpoints on these issues, some noteworthy articles include calculation of p-curve probabilities, the findings of the Open Science’s Reproducibility Project: Psychology, re-analyses of the Open Science data, estimates of questionable research practices, and consideration of both false negatives and false positives. There are many important articles on these issues available. And blog posts on websites surface daily, adding analysis and commentary, along with Facebook groups such as psychMAP.
For science to progress, we need discussion, debate, and challenge—and this is exactly what is happening in social and personality psychology. Although some may think these challenges and discussions reflect something fundamentally wrong with the field, one can also see social and personality psychology as being on the cutting edge of addressing long standing concerns about quality science—something that is also happening across multiple areas of scientific inquiry. Thus, there is good reason to see the current debate and discussion as part of the error correction process called science.
The current discussions are raising new challenges as well as adding urgency to some long-standing concerns with research practices. We are seeing some of the first attempts to do large-scale direct replications of our science, and it is not surprising that there is dispute about the best ways to do this. Consensus on best practices for massive replications are still developing, as are consensus estimates of the replicability of our findings.
New urgency is evident in the critiques of underpowered research that capitalizes on chance findings. Although Jacob Cohen raised this issue in 1962 (see also Rossi, 1990), it gained little traction until recently. Publication bias is another concern that has gained new respect. The gray literature of unpublished studies was until recently the domain of meta-analysts, who developed statistical tests to identify the extent of publication bias and showed how unpublished literature can challenge published findings. Ensuring sufficiently powered studies is long overdue, as is ensuring the accessibility of unpublished and null findings.
So, what is the role of SPSP in these discussions? Members of the SPSP Executive Board have undertaken a number of initiatives to facilitate and promote our science. Several taskforces have worked to identify best practices in the field (e.g., http://spsp.org/task-force-publication-and-research-practices). Our journal policies are set by each editor to reflect her or his sense of how to represent the best science. We continue to alter the format of our annual convention to allow more discussion and innovation. And the Executive Board in 2015 endorsed the spirit of the Center for Open Science guidelines promoting transparency and openness in science.
It is clear that social and personality science (along with other areas of scientific inquiry) can further improve and refine research practices. And we’re optimistic that this improvement is happening. But even seemingly beneficial changes in scientific practice can have unintended negative consequences. For example, although most of us support increasing research power, this practice could favor researchers with access to larger participant pools or encourage study of outcomes assessed with good power and not necessarily ones of scientific importance. Also, although some of us favor of preregistration, others believe that it is not appropriate for much of our research, which is conducted in a more exploratory fashion.
We thus suggest continued dialogue on research practices and continued investigation of the effects on our science. In this way, the field does what we do best—test hypotheses and evaluate data.
So, science progresses on its trajectory of slow, nonlinear advancement. This is possible when we all engage in constructive debate about scientific practice and replicability. Constructive debate is thoughtful, respectful, and focused on the issues, and we recommend that our members focus on these kinds of communications. Only in this way can we continue to refine the science of personality and social psychology.
Wendy Wood, SPSP President, Veronica Benet-Martinez, Elizabeth Haines, Mark Leary, Diane Mackie, Wendy Mendes, Paula Pietromonaco, Toni Schmader, Linda Skitka, Sanjay Srivastava, and Samuel Sommers