Assessment centers have long been held as the golden standard in selection methods. They owe their success to the fact that they are made up of a multitude of exercises, each of which is a valid measure of job performance. It is the additive effect of all of these exercises, technically called incremental validity, that makes assessment centres so good. Organisations frequently invest a great deal of time, money and resources into designing a good assessment centre. Their design typically requires interviews with current job holders, consultation with other stakeholders, exercise design writing time, the piloting of exercises, designing assessor skills training and delivering the assessment center.
To get the best out of this process, much technical expertise is invested in designing relevant exercises, carefully designing the behavioural criteria to be assessed and developing clear, unambiguous rating sheets. Assessors are trained in observation and recording skills, as well as the psychology that can undermine effective assessment. Psychological phenomena such as bias, stereotype, poor memory and harsh or lenient standards can all impact on the accurate assessment of people. A good assessment centre is designed to minimise the impact of all of these, and yet, at the last hurdle, all of this in built objectivity is frequently flushed down the drain. But how can this be? The answer is in the wash-up discussion.
This problem has been recognised for many years, and yet the wash-up discussion is still highly regarded. The reason for this is politics and engagement. Most organisations have rightfully devolved recruitment to line managers with HR acting as a business partner in supporting the process. Line managers want ownership for selection decisions because they know and understand the job roles, and will ultimately be working with the people that are recruited. The “we want to have our say rather than have who we employ thrust upon us" attitude is absolutely right. However, the wash-up discussion is not the place where the selection decision should be made.
As a Business Psychologist I have taken part in many assessment centres where assessors apply themselves diligently to the process. They trust the process, adhere to the indicators, believe in the exercises and are careful to be objective in their evaluations. Yet, during the wash-up, although the numbers may well suggest that the candidate was not right for the organisation somebody pipes up “. . . but they were so good in the customer handling exercise" or “. . . they had real presence", or worse still “. . . but I really like him/her. " Admit it, how many of you have experienced this situation? Now ask yourself, what is actually going on here? Personal bias, recruiting in our own image and all those lovely biases that your training told you to leave at the door are rearing their ugly heads and influencing the selection decision.
The other issue with the wash-up discussion is how the scores are arrived at. Typically, a range of scores exists, and the wash-up discussion is used to arrive at an overall score for a competence. However, the way that this is done is frequently inconsistent not only from centre to centre, but also from candidate to candidate. This inconsistency undermines the validity and reliability of the process. What happens in reality is that different weightings are applied to different exercises, sometimes unconsciously. Sometimes the average score is used, at other times the modal score. However, research has consistently shown that a mathematical combination of scores achieves a higher predictive validity and that this should be the adopted approach for arriving at an overall score.
So, where does this leave the wash-up discussion? The extreme response would be not to have one, but in many cases this would not be politically acceptable. The decision to remove the wash-up discussion in one organisation was initially met with resistance. This resistance was dealt with by explaining that managers were still involved in the assessment process - they observed candidates during exercises and provided ratings. This involvement generated the data that would determine if a candidate should pass or fail. However, where the wash-up discussion cannot be removed altogether, you should at least introduce the following:
- Nominate an impartial chair person for the wash-up discussion.
- Calculate overall scores mathematically, either by simply adding them, or calculating the average.
- Have a clearly defined pass/fail mark.
- Where a candidate is borderline or has performed inconsistently, revisit the scores. The chair should facilitate a discussion whereby the relevant assessors provide clear evidence for why they have scored the person the way that they have. This should focus on behavioural evidence only. Once you have satisfied yourselves with the individual scores, apply the pass/fail mark to make the decision.
- Finally, use the wash-up discussion to collect key feedback messages - what was the person best at? What were they least good at? Use this information to provide feedback to successful and unsuccessful candidates.
Mike Idziaszczyk is a Psychologist at Pearn Kandola Business Psychologists based in Oxford. He can be contacted on 44 1865 516202 or emailed at email@example.com