We use multi-choice questions in Modlettes because it is easy to collect and record the results. They can also be used for competency checks in simulations which is very important for teaching soft skills. Some people argue that multi-choice questions shouldn’t be used because they only measure recall of content.
That’s like saying PowerPoint is bad because the slides are cluttered with content. That is not the fault of PowerPoint but of the person designing the slides! Multi-choice questions that only measure recall are the fault of the multi-choice question writer.
If you want to become an expert and create multi-choice questions that lead to great learning then take some notice of the following flaws that commonly occur.
Answer choices are not all the same length.
Most experienced test takers do what they can to eliminate some of the answer choices, especially when they are unsure. This gives them a better chance of guessing the right answer.
And, they know that the longest answer is often the correct one. That’s because we often include more detail in the correct answer. If you have ever watched the TV quiz show “The Chase” you will notice that the chasers often use this explanation for a choice when they don’t know the answer.
Example: You are managing a production unit with a staff of 18. One of your team members is always showing off what she knows and the other members have become disdainful of her. You want a cohesive team. What should you do?
(a) Talk to the other members about their attitude to her
(b) Call her into your office and face her with the facts and the effect her attitude is having on the other members of you team.
(c) Fire her.
The correct answer is (b). It is also the longest and has the most detail.
How to Get It Right: Write all answer choices so they are about the same length. If this is too hard consider writing two answers that are shorter and two answers that are longer.
Including unnecessary details
Questions should be focused, concise, and clear. If they include unnecessary information, people will try their hardest to make sense of how that information relates to the answer choices. This increases the mental effort of the question in unnecessary ways and diminishes the question’s value.
Example: You want to use your charcoal kettle for warmth in your gazebo on chilly evenings and to roast marshmallows. Can you safely operate the charcoal burner under the gazebo roof?
(a) Yes, if the gazebo roof is high enough for the heat to dissipate
(b) Yes, if you use screening material of charcoal container to reduce sparks
(c) No, because the smoke and fumes may damage the gazebo ceiling
(d) No, because only gas fired barbecues can be used in a gazebo.
The correct answer is (a), and it is the only answer that correctly answers how to safely operate a cooking device in a gazebo. The unnecessary detail about warmth and roasting marshmallows confuses the question.
How to get it right: Keep all answer choices similar in length and level of detail.
Using, “All of the above” and “None of the above” as answer choices
Example: Which of the following are good woods to use in a wood burner?
(Select the correct answer)
(a) Oak
(b) Mackracarpa
(c) Pine
(d) All of the above
The correct answer is (d). The all or none may seem an easy way to add another answer, but they are poor choices. If someone can identify a single answer choice as correct, they know that “none of the above” isn’t correct. If they can identify a single choice as incorrect, they know that “all of the above” isn’t correct. When people can eliminate answers, they can more easily get the question correct, even if they don’t know the answer.
How to get it right: Don’t use “All of the above” or “None of the above” as answer choices.
I hope these thoughts will help you avoid the pitfalls most beginners fail to negotiate.