SIPS Commendation Award 2024 for more transparency in the preprocessing of reaction times

The Society for the Improvement of Psychological Science (SIPS) aims at improving methods and practices in psychological research. Since 2020 the SIPS Commendation program recognizes projects that support this mission. This year, the Reaction Time Preprocessing Initiative received such a Commendation for its efforts to reach consensus on the preprocessing of reaction time data.

In the behavioral, cognitive and social sciences, reaction time has always been an important source of information. However, analyses of reaction time data are influenced by analytical choices of the researcher and the order in which these decisions are made. In recent years, there have been various efforts to standardize data preprocessing of complex neural data, while there is still no common preprocessing pipeline (or even documentation and reporting) for behavioral measures such as reaction times. However, this is essential for the reproducibility and replicability of research results. The Reaction Time Preprocessing Initiative reached a first milestone with the development of a checklist for reporting reaction time preprocessing and lays a foundation to clarify related decisions, improve transparency and promote best practices in this area. The checklist is meant to facilitate transparent reporting of reaction time preprocessing for research, editing, and teaching. It is based on a systematic literature review, an expert consensus survey, and a multiverse analysis.

The Reaction Time Preprocessing Initiative is an international and interdisciplinary group of researchers that emerged from the (Too) Many Shades of Reaction Time Data Pre-Processing hackathon at SIPS 2021 conference. A total of 16 researchers from 7 countries (Germany, the Netherlands, Great Britain, Serbia, USA, Austria, Switzerland) at 13 universities are involved. They share a common motivation for good psychological science practice and the realization that there is still no consensus on how to evaluate even basic psychological measures.

Related publication:

Loenneker et al. (2024). We don't know what you did last summer. On the importance of transparent reporting of reaction time data pre-processing. Cortex, 172, 14-37, https://doi.org/10.1016/j.cortex.2023.11.012.

SIPS Awards Announcement:

https://improvingpsych.org/2024/08/