The purpose of a test is to determine the learner’s mastery of learning or performance objectives. Tests should also show whether a learning experience was successful. I am a great believer in how useful good multichoice quiz questions can be in determining the effectiveness of a training intervention.
Effective distractors (the incorrect answers in multichoice questions) can play an important role in getting the measurement data you need. Well written distractors provide information that no other aspect of assessment can. However, writing effective distractors is one of the most difficult tasks of learning design.
Distractors Provide Valuable Information : The purpose of distractors is to see if the learner can discriminate between correct and incorrect answer choices. This demonstrates what a participant knows or doesn’t know. It shows whether a person performs a task correctly or incorrectly. Effective distractors have the potential to let us know whether:
- A test taker has achieved the performance objective
- The individual has misconceptions, faulty reasoning or will make errors in the workplace.
- The particular question needs improvement
- A corresponding portion of a learning experience is ineffective.
Writing Good Distractors : Writing good distractors can be time and effort consuming, even frustrating. Here are some ideas to help you make distractors more useful.
Use content experts: Rather than struggle with unfamiliar content, partner with subject matter experts where possible.
Consider these questions: Experts suggest considering the types of questions below:
- What do people usually confuse this concept with?
- What mistakes do people make when performing this task?
- What is a common error people make when solving this problem?
- What are common misconceptions in this field?
Use Similarity to Create Great Distractors : Another approach is to start the distractor writing process by identifying the key feature(s) in the correct answer. Then systematically remove or replace one of the key features and replace with another one to create the distractor.
Distractors Are Not to Trick the Learner : Trainers are agreed that a distractor should not attempt to trick the learner. That will only provide poor information to the person writing the assessment. However, if the distractor is based on common errors, it will appear attractive.
Similar Distractors are the Strongest : Many studies found that answer choices with distractors that were similar to the correct answer increased the difficulty of the test item. It is more difficult for test takers to discriminate between items that are parallel than items that are dissimilar.
Content similarity can be based on many different aspects of a distractor, including:
- Content (similar concept or idea)
- Similarity in numeric options (for calculations)
The Modlettes’ system provides three options for multichoice questions, a learning quiz with comment on all answers, a test quiz where results are sent back to the supervisor, and a written answer. These three options give a very useful instrument for checking performance and knowledge. www.modlettes.com
We recognise the research of Connie Malamed for the concepts in this article.