Exercising judgement is a critical factor in most jobs today. We refer to these assessments as Situational Judgement Assessments. Using multi-choice questions enables us to move beyond assessments of what people know (knowledge of what) to assessments of how that knowledge will be applied in the workplace (Knowledge of how). Judgement is not the preserve of the CEO – it is a competency or skill that reaches all levels of an organization, We can’t be sure somebody has really learnt a skill until they can demonstrate its use.
One of the little used methods of testing and assessment of judgement or decision making is the scenario based multi-choice question.
Situational judgement is the ability to draw sound conclusions or make sound predictions about outcomes in a given situation.
A strong reason to consider SJAs is the opportunity for trainers to extend the reach of their assessment beyond the evaluation of what someone knows. Knowing is not the same as doing. SJAs offer a vehicle to assess for higher-level skills and more complex learning objectives.
Consider Bloom’s taxonomy which many of you will be familiar with.
Create - Product new or original work
Evaluate - Justify a stand or decision
Analyse - Draw connections among ideas
Apply - Use information in new situations
Understand - Explain ideas or concepts
Remember - Recall facts and basic concepts
SJA’s clearly show that this form of assessment offers the opportunity to move beyond the lower levels of this model, “remember” and “understand”, and develop assessments that cover the higher levels including “analyse” and “evaluate”.
This opportunity is particularly important for certification programmes and testing competency before hiring. It enables practitioners to ensure that their assessment of competence is not limited to what people know but extends to assessing what people are likely to do in the workplace – so that their decisions and actions demonstrate appropriate use of knowledge through effective judgement.
Consider the following Example:
Type 1: What does a yellow traffic light mean?
Type 2: If you are driving toward an intersection and the light turns from yellow to red, what should you do?
Type 3: John is driving a close friend to an appointment and they are running 15 minutes late. They are nearing a controlled intersection with traffic lights. John and his friend can see the intersecting road to some extent and there seem to be no cars on it. The light turns to yellow as John approaches. The car behind sounds their horn and appears to be accelerating in the expectation that John is going to cross the intersection. John’s friend tells him to put his foot down, as he is running late and there are no cars coming. What should John do?
Type 1 is a knowledge question and relies on memory of facts. Type 2 is a question about the application of knowledge and requires some judgement. Type 3 is a question about making a judgement that involves evaluating a situation that has numerous factors (the road layout, traffic light conditions, personal circumstances, other actors involved in the situation) and calls for a choice between more – or less – appropriate responses to that situation.
What is an SJA
An SJA is an assessment or test in which participants are presented with work-relevant situations and asked to make judgements about possible responses. SJA’s can be used to assess responses in soft skills situations, such as communication, empathy and conflict resolution.
Some examples of judgements that SJA’s are suitable for:
The Structure of an SJA item
Let’s explore this structure in an example using a technical team.
Robyn works as part of a technical support team that produces work internally for an organization. She has noticed that often work is not performed correctly or a step has been omitted from a procedure. She is aware that some individuals are more at fault than others as they do not make the effort to produce high quality results and they work in a disorganized way. What would you advise Robyn to do to make the most effective responses to this situation?
Option 3 is a way to deal with the errors but places a significant amount of work on the participant.
It also does not address the behavior of the other members of the team. Option 2 seems reasonable but will not address the issues immediately (access to another team to be arranged) and may not address the team’s disorganised approach. These two options are unlikely to make the situation worse, but they are not guaranteed to address the team’s behavior.
Option 4 is asking a disorganised team to engage in an open and undefined set of experiments that could increase rather than reduce errors in the work produced. This is, on balance, the least effective of the options presented.
Option 1 does require some confidence in dealing with the potential push-back from the other team members. However, it does address the team’s behavior directly and it does approach the problem in terms of consequences for the team as a whole rather than as a subjective criticism of the team and any individual, This, therefore, is the option that is most likely to have a positive effect.
There are other ways in which we can use the Modlettes multi-choice questions for situation judgement assessments, and I will expand on these in a later publication. Meanwhile, hop onto a free trial at www.modlettes.com and have a look at how the quiz functionality can be used. We are very proud of how flexible our quiz section can be.
