An evaluation of the relative value of clinical decision-making tests, situational judgement tests, and selection centres, in the recruitment of trainee doctors in the UK has now been presented in two new papers

Students sat in an exam hall working on a paper

These comprehensive papers explore how selection into training for General Practice (Tiffin et al., 2024; ref 1) and Psychiatry (Tiffin et al., 2024; ref 2) medicine has evolved over the past years, including changes made reflecting the constraints imposed by COVID-19. Topics discussed include the different predictive power of each selection component used, the sensitivity of each assessment to the influence of applicant demographic profile, and how to deal with missing data when conducting analysis effectively.  

Nandini Chakraborty, National Lead for Recruitment in Psychiatry, comments “It is crucial that we keep reviewing recruitment processes so that they continue to be fair and logistically deliverable. This paper is a start, an example of our probing into what is working and what might be done better, what needs to be explored more deeply. We hope to keep exploring various aspects of recruitment processes, learn and improve.’

The article focused on GP selection adds to previous research in this area (Patterson et al, 2013; ref 3, Botan et al., 2022; ref 4, Siriwardena et al, 2023; ref 5) by completing a thorough review of how predictive the clinical problem-solving test, situational judgement test (SJT), and selection centre, are of performance in the GP qualifying exams (required for successful completion of training). 

Evidence shows that the machine-marked clinical test and SJT are good predictors of success in training. The selection centre adds value in helping to inform decisions for those candidates who achieve a low, but passing, score in the tests. This could be a useful strategy to ensure the number of GP trainees is maximised, whilst maintaining key standards of practice.

No such research has been published exploring Psychiatry selection thus far: This paper is the first to provide evidence that the combination of a clinical problem-solving test, an SJT, and a selection centre for borderline applicants, is an effective solution to identify those who will successfully complete the postgraduate clinical examinations during the Psychiatry training process.

Key points to consider from both articles include:

  1. The value of using machine-marked tests, focused on skills such as clinical decision-making and understanding the desirable professional attributes at work (empathy, professionalism, coping with pressure), to accurately identify a large proportion of those applicants that will succeed in training programmes
  2. The added value of using a selection centre to inform decisions for more borderline applicants
  3. The possibility that well-designed, reliable, selection centres could aid in “widening access” to medical training as they are less sensitive to the influence of demographic variables (gender, ethnicity, place of medical training)
  4. The use of path analysis to understand exactly how each selection component adds validity to overall decision-making (aiding decisions about sequencing and utility of each component in the selection process)
  5. The use of multiple imputation to deal with missing data (a common problem when conducting validity studies on medical selection due to range restriction from selection decisions, and lengthy delays before outcome data is available).

The articles reinforce past research findings about the benefits of using SJTs and MMIs in high-stakes medical selection (Cleland et al., 2023; ref 6), and identify the importance of regularly reviewing processes as selection processes or strategy change in response to external demands and constraints. Please feel free to get in touch or post a response if you would like to share examples or reflections, or discuss challenges within your selection process. 

Find the full articles here:  Link to GP Paper

Link to Psych paper

References:

  1. Tiffin, P. A., Morley, E., Paton, L. W., & Patterson, F. (2024). New evidence on the validity of the selection methods for recruitment to general practice training: a cohort study. BJGP open.
  2. Tiffin, P. A., Morley, E., Paton, L. W., Chakraborty, N., & Patterson, F. (2024). The validity of the selection methods for recruitment to UK core psychiatry training: cohort study. BJPsych bulletin, 1-10.
  3. Patterson, F., Lievens, F., Kerrin, M., Munro, N., Irish, B. (2013). The predictive validity of selection for entry into postgraduate training in general practice: evidence from three longitudinal studies. British Journal of General Practice, 63(616), e734-741.
  4. Botan V, Williams N, Law GR, Siriwardena AN. (2022) How is performance at selection to general practice related to performance at the endpoint of GP training? University of Lincoln.
  5. Siriwardena, A.N., Botan, V., Williams, N., Emerson, K., Kameen, F., Pope, L., Freeman, A., Law, G. (2023). Performance of ethnic minority versus White doctors in the MRCGP assessment 2016-2021: a cross-sectional study. British Journal of General Practice, 73(729), e284-293.
  6. Cleland, J., Blitz, J., Cleutjens, K. B., Oude Egbrink, M. G., Schreurs, S., & Patterson, F. (2023). Robust, defensible, and fair: The AMEE guide to selection into medical school: AMEE Guide No. 153. Medical Teacher, 45(10), 1071-1084.