Exercising judgement is a critical factor in most jobs today. We refer to these assessments as Situational Judgement Assessments. Using multi-choice questions enables us to move beyond assessments of what people know (knowledge of what) to assessments of how that knowledge will be applied in the workplace (Knowledge of how). Judgement is not the preserve of the CEO – it is a competency or skill that reaches all levels of an organization, We can’t be sure somebody has really learnt a skill until they can demonstrate its use.
One of the little used methods of testing and assessment of judgement or decision making is the scenario based multi-choice question.
Why Assess for Situational Judgement
Situational judgement is the ability to draw sound conclusions or make sound predictions about outcomes in a given situation.
A strong reason to consider SJAs is the opportunity for trainers to extend the reach of their assessment beyond the evaluation of what someone knows. Knowing is not the same as doing. SJAs offer a vehicle to assess for higher-level skills and more complex learning objectives.
Consider Bloom’s taxonomy which many of you will be familiar with.
Create – Product new or original work
Evaluate – Justify a stand or decision
Analyse – Draw connections among ideas
Apply – Use information in new situations
Understand – Explain ideas or concepts
Remember – Recall facts and basic concepts
SJA’s clearly show that this form of assessment offers the opportunity to move beyond the lower levels of this model, “remember” and “understand”, and develop assessments that cover the higher levels including “analyse” and “evaluate”.
This opportunity is particularly important for certification programmes and testing competency before hiring. It enables practitioners to ensure that their assessment of competence is not limited to what people know but extends to assessing what people are likely to do in the workplace – so that their decisions and actions demonstrate appropriate use of knowledge through effective judgement.
Consider the following Example:
Type 1: What does a yellow traffic light mean?
- Stop
- Go
- Caution
- Look behind you
Type 2: If you are driving toward an intersection and the light turns from yellow to red, what should you do?
- Speed up and cross the intersection
- Continue at the same speed and cross the intersection
- Stop suddenly
- Stop gradually
Type 3: John is driving a close friend to an appointment and they are running 15 minutes late. They are nearing a controlled intersection with traffic lights. John and his friend can see the intersecting road to some extent and there seem to be no cars on it. The light turns to yellow as John approaches. The car behind sounds their horn and appears to be accelerating in the expectation that John is going to cross the intersection. John’s friend tells him to put his foot down, as he is running late and there are no cars coming. What should John do?
- The road appears to be clear, so speed up and cross the intersection to get through it as soon as possible
- Stay at the same speed and continue to drive through the intersection while keeping an eye out for crossing cars
- The light is about to turn red, so you should stop immediately and ignore the car behind you as they will need to stop as well.
- Ignore the friend’s irritation and move to the side of the road and let the other car past.
Type 1 is a knowledge question and relies on memory of facts. Type 2 is a question about the application of knowledge and requires some judgement. Type 3 is a question about making a judgement that involves evaluating a situation that has numerous factors (the road layout, traffic light conditions, personal circumstances, other actors involved in the situation) and calls for a choice between more – or less – appropriate responses to that situation.
What is an SJA
An SJA is an assessment or test in which participants are presented with work-relevant situations and asked to make judgements about possible responses. SJA’s can be used to assess responses in soft skills situations, such as communication, empathy and conflict resolution.
Some examples of judgements that SJA’s are suitable for:
- How a team member interprets and understands a typical work situation in terms of the critical aspects of a situation they should focus on or prioritise
- The correct application of job knowledge in terms of which actions are more or less likely to achieve a task objective or a desired outcome
- How best to work with and interact with other employees such as members of a work group or team to resolve an issue and overcome obstacles to achieving a shared goal or outcome
- How to deal with potential dilemmas, particularly those where outcomes are less clear or where information is more ambiguous
- Which responses to typical work situations are more versus less aligned with the expectations of a company, such as compliance with company values and organization norms for behavior.
The Structure of an SJA item
Let’s explore this structure in an example using a technical team.
Robyn works as part of a technical support team that produces work internally for an organization. She has noticed that often work is not performed correctly or a step has been omitted from a procedure. She is aware that some individuals are more at fault than others as they do not make the effort to produce high quality results and they work in a disorganized way. What would you advise Robyn to do to make the most effective responses to this situation?
- Explain to your team why these procedures are important and what the consequences are of not performing these correctly.
- Try to arrange for your team to observe another team in the organization who produce high quality work
- Check your own work and that of everyone else in the team to make sure any errors are found.
- Suggest that the team tries many different ways to approach their work to see if they can find a method where fewer mistakes will be made.
Option 3 is a way to deal with the errors but places a significant amount of work on the participant.
It also does not address the behavior of the other members of the team. Option 2 seems reasonable but will not address the issues immediately (access to another team to be arranged) and may not address the team’s disorganised approach. These two options are unlikely to make the situation worse, but they are not guaranteed to address the team’s behavior.
Option 4 is asking a disorganised team to engage in an open and undefined set of experiments that could increase rather than reduce errors in the work produced. This is, on balance, the least effective of the options presented.
Option 1 does require some confidence in dealing with the potential push-back from the other team members. However, it does address the team’s behavior directly and it does approach the problem in terms of consequences for the team as a whole rather than as a subjective criticism of the team and any individual, This, therefore, is the option that is most likely to have a positive effect.
There are other ways in which we can use the Modlettes multi-choice questions for situation judgement assessments, and I will expand on these in a later publication. Meanwhile, hop onto a free trial at www.modlettes.com and have a look at how the quiz functionality can be used. We are very proud of how flexible our quiz section can be.