A new publication by Professor Steve Woods, University of Surrey, and Professor Fiona Patterson, Director of Work Psychology Group, reviews the use of cognitive ability testing for access to graduate and higher professional occupations, promoting further debate and reflection for both academics and practitioners.
The article goes beyond previous reviews by shedding light on issues surrounding socio-economic status (SES) and socio-inequalities in particular. The role of cognitive ability tests in selection to the professions and their impact on applicants from more disadvantaged socio-economic backgrounds is relatively neglected within research.
Fiona comments, “the reasons behind these observed differences for those in different socio-economic groups are complex and multi-faceted, and our article reviews the issues in detail to promote further debate on this topic. Considering the latest research evidence, rather than focus on critiquing of any particular test or test publisher, our paper questions how cognitive ability tests are actually used in recruitment into professional and graduate occupations, which are often heavily weighted in selection”.
This article (Woods & Patterson, 2023) follows a new meta-analytic study by Sackett et al. in 2022 that disrupts the way that we think about the relative validity of selection methods, in particular, cognitive tests. Steve and Fiona expand on this work, considering five key points during their article;
- Evidence of adverse impact in cognitive test outcomes;
- The tendency to position cognitive ability testing early in selection processes in high-volume recruitment;
- Recent evidence challenging the relative validity of cognitive ability tests;
- Weaknesses in historical primary validity studies;
- Conceptually flawed examination of differential validity evidence in the literature.
In reviewing the research over many decades and exploring the issues, the authors offer two scenarios to encourage readers to reflect upon how cognitive ability tests are used within selection processes. The first is to consider reducing the weighting given to cognitive tests compared to other selection tools and/or re-positioning such tests to come later in selection processes. The second is to consider replacing cognitive tests as an early screening tool and to explore the use of other selection methods to reduce the diversity impact.
Regarding directions for future research to further elaborate on this topic, Steve comments, “Our review points to a need for psychologists to develop better and more nuanced understanding of the effects of contextual and socio-economic background variables on assessment outcomes and validity. More broadly, our discussion underlines the importance of maintaining our evidence base for the validity of different selection methodologies, reflecting the demands of contemporary work and organisations. Pursuing these research lines will require innovation and interdisciplinary collaboration by psychologists in the coming years.”
The authors acknowledge that whilst their paper raises some critical points, there isn’t a clear solution. This raises the question, if you were to re-configure, re-weight or remove cognitive ability tests from your selection process, what would you do instead? We would welcome your views in this debate so please get in touch or post a response if you would like to discuss further.
Find the full article here: A critical review of the use of cognitive ability testing for selection into graduate and higher professional occupations (wiley.com)
References:
Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. Journal of Applied Psychology, 107(11), 2040.
Woods, S, A. & Patterson, F. (2023). A Critical Review of the use of Cognitive Ability Testing for Selection into Graduate and Higher Professional Occupations. Journal of Occupational and Organizational Psychology. https://doi.org/10.1111/joop.12470
