Measuring Personality of Large Language Models: Concepts, Methods, and Improvements

Abstract: With the widespread use of large language models in human computer interaction, their personality related manifestations have become important factors shaping user experience, trust judgments, and safety governance. However, the concept of personality in large language models remains underdefined, and existing assessment approaches are fragmented and lack systematic psychometric standards. This article clarifies the conceptual meaning and generative mechanisms of personality in large language models, reviews the theoretical frameworks underlying its assessment, and classifies existing methods into questionnaire responses, linguistic cues, and situational behaviors. It further identifies major challenges, including limited fit between measurement tools and tasks, insufficient psychometric evidence, weak cross method comparability, and inadequate foundations for score interpretation. Future research should develop personality frameworks and measurement tools tailored to large language models, incorporate modern psychometric models, advance adaptive and dynamic assessment, and strengthen reliability and validity validation, standardized evaluation, and norm construction. These efforts can promote more scientific and standardized assessment of personality in large language models.