Writing Multiple-choice Test Items

• Writing Item Stems
  
  
• Ask questions that require more than knowledge of facts
  
  
• a question might require selection of the best answer when all of the options contain elements of correctness. Such questions tend to be more difficult and discriminating than questions that merely ask for a fact. (Frary, 1995)
  
  
• In general, the stem should not pose more than one problem, although the solution to that problem may require more than one step. (Kehoe, 1995)
  
  
• Construct the stem to be either an incomplete statement or a direct question, avoiding stereotyped phraseology
  
  
• rote responses are usually based on verbal stereotypes (Kehoe, 1995)
  
  
• Include as much information in the stem and as little in the options as possible.
  
  
• For example, if the point of an item is to associate a term with its definition, the preferred format would be to present the definition in the stem and several terms as options rather than to present the term in the stem and several definitions as options. (Kehoe, 1995)
  
  
• Restrict the use of negatives in the stem.
  
  
• Negatives in the stem usually require that the answer be a false statement. Because students are likely in the habit of searching for true statements, this may introduce an unwanted bias.
  
  
• Avoid irrelevant clues to the correct option
  
  
• Grammatical construction, for example, may lead students to reject options which are grammatically incorrect as the stem is stated. (Kehoe, 1995)
  
  
• Use simple and clear language in the questions.
  
  
• If the language is difficult, students who understand the material but who do not have strong language skills may find it difficult to demonstrate their knowledge.
  
  
• Writing Options
  
  
• Be satisfied with three or four well constructed options. (Kehoe, 1995)
  
  
• Ask questions with varying numbers of options.
  
  
• There is no psychometric advantage to having a uniform number, especially if doing so results in options that are so implausible that no one or almost no one marks them (Frary, 1995)
  
  
• Make incorrect alternatives attractive to students who have not achieved the targeted learning objectives (Childs, 1989)
  
  
• Instead of repeating words in each alternative, include these words in the main body of the question.
  
  
• This will make the question easier to read and the options easier to compare.
  
• The grammar or structure of the main part of the question must not contain clues to the correct response, however. (Childs, 1989)
  
  
• Don't put negative options following a negative stem
  
  
• Empirically (or statistically) such items may appear to perform adequately, but this is probably only because brighter students who naturally tend to get higher scores are also better able to cope with the logical complexity of a double negative. (Frary, 1995)
  
  
• Try to avoid use "all of the above."
  
  
• Recognition of one wrong option eliminates "all of the above"
  
• recognition of two right options identifies it as the answer, even if the other options are completely unknown to the student. (Frary, 1995)
  
  
• Options which read “none of the above,” “both A and E above,” “all of the above,” etc., should be avoided when the students have been instructed to choose the “best answer,” which implies that the options vary in degree of correctness (Kehoe, 1995)
  
  
• Do ask questions with "none of the above" as the final option, especially if the answer requires computation.
  
  
• Its use makes the question harder and more discriminating, because the uncertain student cannot focus on a set of options that must contain the answer.
  
• Of course, "none of the above" cannot be used if the question requires selection of the best answer
  
• It should not be used following a negative stem. (Frary, 1995)
  
  
• Make all choices exactly parallel.
  
  
• Novice test writers tend to make the correct answer longer and more carefully worded and, by doing so, may provide a clue to the correct answer. (Childs, 1989)
• Construct distractors that are comparable in length, complexity and grammatical form to the answer, avoiding the use of such words as "always," "never," and "all.“ (Kehoe, 1995)
  
  
• After the options are written, vary the location of the answer on as random a basis as possible.
  
  
• A convenient method is to flip two (or three) coins at a time where each possible Head-Tail combination is associated with a particular location for the answer. (Kehoe, 1995)
  
  
• If possible, have a colleague with expertise in the content area of the exam review the items for possible ambiguities, redundancies or other structural difficulties.
  
  
• Having completed the items we are typically so relieved that we may be tempted to regard the task as completed and each item in its final and permanent form. Yet, another source of item and test improvement is available to us, namely, statistical analyses of student responses. (Kehoe, 1995)

Click here to view an interactive guide to writing multiple choice test items.

Source:

Unless otherwise specified, the texts were from Popham, W. James (2002). Classroom assessment: What teachers need to know (3rd ed.). MA: Allyn & Bacon


Other References:

Childs, Ruth A. (1989). Constructing classroom achievement tests. ERIC Digest ED315426.

Frary, Robert B. (1995). More multiple-choice item writing do’s and don’ts. ERIC Digest ED398238.

Kehoe, Jerard (1995). Writing multiple-choice test items. ERIC Digest ED398236.