WPG works in assessment for selection and development and we have reflected on learnings from our field that are relevant to this situation. The challenges of grading students when key assessment data (exams) is not available, is similar to challenges in recruitment at the moment.
So, what might be important to consider when thinking about the impact of assessment?
We’ve reflected on topics including bias in assessment, disproportionate impact on certain groups, perceptions of fairness, and the need for ongoing support.
No single method of assessment is completely accurate and all assessments have the potential to favour certain groups over others (even Artificial Intelligence!). We need to dispel the myth that any one method (be that exams or interviews) will produce an ‘objective’ result.
Combining different kinds of assessment and methods is often a fairer way to establish performance or potential. Rather than relying on a single measure, consider how different assessments when combined can give a fairer and more holistic measurement.
Nicola Sturgeon said adjustment (moderation) of Scottish grades was “too focused on the overall system, not enough on individual pupils”. In aiming to replicate the spread of results that has been found in previous years, it was assumed that the ‘normal’ way of assessing students is fair, which is not necessarily the case.
Whilst many grades were initially moderated down in Scotland, this specifically impacted students in more deprived areas. Pass-rates reduced by 12.5% in those areas, compared to only 6.9% in more affluent areas.
The same pattern is being predicted in England: students attending schools with variable or low exam results in the past are at risk of being the worst affected. We know this is likely to have a particularly negative impact on those from Black, Asian and Minority Ethnic backgrounds.
It should be carefully considered if outcomes like this are justifiable. We will never reach our stated goal of widening access to education if processes rely primarily on exam results analysed in different ways.
What about recruitment?
Recruitment has been through a similar journey. If applicants are selected on past experience or academic record alone, we know this is likely to disadvantage some candidates unfairly because they have not had equitable access to opportunities.
Face validity – “does this feel like a fair way of making decisions” – is also critical to acceptance. If ‘the usual way’ of making decisions is not possible, it is likely that people will challenge if the ‘new’ way of doing things is comparable to the ‘usual’ way. Perceptions matter.
Diversity of views is good. Explore perceptions of fairness before implementing a process and work with those involved to consider alternative ways of doing things. Consider how communication of processes can be as clear, honest and non-ambiguous as possible.
Eliminating inequitable outcomes such as these takes time. In the meantime, recognise some groups will experience more difficulties. How can they be given access to the same opportunities and support that others receive? How could the gap be closed?
Researchers from Birmingham and Nottingham Universities surveyed 500 candidates about their experiences of taking A levels this year and found that ; while 82% of white pupils were satisfied with how their school managed the crisis, 67% of black pupils and only 42% of Asian pupils felt the same”.
Conversations like this can be uncomfortable and challenging. However, we need to take responsibility for making sure our assessments & assessment processes are designed and used in a fair and inclusive way and be open to making change if the data shows otherwise.
Would you be interested in an initial no-obligation conversation about your organisation’s selection and assessment processes and how we could work with you to review them for fairness? Do get in touch– we’d love to hear from you!
