Measurement Reliability of Cognitive Tasks: Current Trends and Future Directions

Abstract: Cognitive tasks, as a core method in cognitive psychology, have recently been used to measure individual cognitive differences. However, the estimation methods and current research on the reliability of cognitive tasks have received relatively little attention. Based on the traditional concept of reliability, this paper identifies common misunderstandings in researchers’ comprehension and assessment of cognitive task reliability and summarizes two major challenges in evaluating it. By reviewing and summarizing the computational methods and selection of reliability indicators, this paper further explores three key strategies for improving cognitive task reliability: increasing inter-individual differences, reducing error variance, and refining computational and statistical methods. Through a systematic review of conceptual misunderstandings, appropriate evaluation methods and indicator selection, major challenges, and reliability enhancement strategies, this paper aims to provide researchers with suitable methodological choices and practical applications.