Does Mental Ability Affect Question Interpretation on Personality Tests?

[We’re pleased to welcome Amy DuVernet who was the corresponding author on the article “General Mental Ability as a Source of Differential Functioning in Personality Scales” from Organizational Research Methods.]

Individuals vary on a number of characteristics. Our ability to accurately measure their standings on those characteristics is pivotal to our understanding of individual differences and the drivers of individual behavior. Our study focused specifically on the interaction between personality measurement and intelligence (i.e., general mental ability). We utilized Item Response Theory techniques to examine differences in item characteristics across groups of varying levels of general mental ability. In other words, we investigated whether intelligence plays a role in the way an individual interprets and responds to questions designed to gauge personality traits, such as extraversion and conscientiousness.

A person high in intelligence may be better able to interpret and thus respond to a personality item if that item uses complex language or requires a great deal of cognitive processing. For example, the International Personality Item Pool (Goldberg, Johnson, Eber, Hogan, Ashton, Cloninger, & Gough, 2006) item “I shirk my duties” requires respondents to understand the meaning of the relatively uncommon term “shirk”, to recall instances of shirked work duties, and to gauge how those recollections map onto the response options (e.g., strongly agree to strongly disagree).

Our results confirmed that, while most personality items did not demonstrate significantly different characteristics across groups, certain items are indeed interpreted differently by individuals with highly different intelligence levels. For example, all negatively keyed items (i.e., items in which strong endorsement indicates less of the underlying trait being measured) exhibited differential item functioning, suggesting that respondents with low cognitive ability interpreted and responded to these items differently than those with high cognitive ability. These findings have implications for the construction of personality and other noncognitive measures. Ideally, the measurement of these constructs should not be influenced by individuals’ intelligence; however, the results of this study indicate that intelligence can influence the response process for non-cognitive measures.

Click here to read “General Mental Ability as a Source of Differential Functioning in Personality Scales” from Organizational Research Methods. Make sure to sign up for e-alerts by clicking here and stay up to date on all the latest from Organizational Research Methods!

 Amy M. DuVernet is the Director of Corporate Research at Training Industry, Inc, where her work focuses on learning and development research to inform best practices. She earned her PhD in Industrial and Organizational Psychology from North Carolina State University.

Natalie Wright is an assistant professor in the Department of Psychology and Counseling at Valdosta State University. She earned her PhD in industrial and organizational psychology from North Carolina State University in 2013. Her current research focuses on the psychometric evaluation of psychological measurements.

Adam W. Meade is Professor of Psychology at North Carolina State University. His interests relate to the application of quantitative methods in organizational research in novel approaches to psychological measurement. He serves on various editor boards and as Associate Editor for Organizational Research Methods.

Chris Coughlin is a Senior Research Scientist on the Product Development and Innovation team at CEB. In this role, he leads the development, validation, and implementation of call center, software, and computer skill simulations. Prior to joining CEB, he worked on organizational development initiatives at Spherion, a Randstad company. He earned his BS in Psychology from the University of Georgia and his MS in Industrial-Organizational Psychology from Valdosta State University. He is a member of the Society of Industrial and Organizational Psychology, Society for Human Resource Management, and the American Psychological Association.

Tracy M. Kantrowitz is Vice President of Research and Development at CEB’s SHL Talent Measurement Solutions. In this role, she is responsible for the development of assessment content and research related to employee selection. Dr. Kantrowitz has published in leading journals and presented at national conferences on topics such as predictors of job performance, computer adaptive testing (CAT), and unproctored internet testing (UIT). Dr. Kantrowitz holds a PhD in industrial/organizational psychology from the Georgia Institute of Technology.