Serdar Balcı

“The Design and Use of Multiple Choice Tests”

Multiple Choice Exams

“ Written properly, multiple choice exams correlate strongly with assessments by descriptive tests ”

Brown, Robert, “Multiple Choice Versus Descriptive Tests”, Frontiers in Education Conference, Reno NC 2001.

Use of Multiple Choice Exams

Multiple Choice vs. Descriptive Examinations



Setting the exam

Grade Consistency

Structure of a Multiple Choice Question



Formulating Good Multiple Choice Questions


Answer Options

Single correct answer options

Collective answer options:

__ “ (a) and (c) are both correct ”__

__ “ all of the above are correct ”__

__ “ none of the above are correct ”__

Plausible Answer Strategies

Good questions contain several plausible answer options

Grey area between designing “tricky” questions and providing answer options that test knowledge thoroughly.

**Inadvertently giving away the answers **

Giving away the answer…

A verbal question:

__ The forbidden fruit that Eve offered Adam was an:__

__ (a) apple (b) pear__

__ (c) banana (d) melon__

__ (e) carrot__

i.e. a numerical question:

__ It costs a vendor $1.00 each time s/he sells a hot dog to a customer. If the price of the hot dog is $3.00, what is the contribution margin percentage?__

__ (a) 17.5% (b) 22.5%__

__ (c) 47.5% (d) 66.7%__

__ (e) 82.0%__

“Plausible” Answer Strategy

__ It costs a vendor $1.00 each time s/he sells a hot dog to a customer. If the price of the hot dog is $3.00, what is the contribution margin percentage?__

__ (a) 300% (b) 200%__

__ (c) 67.7% (d) 50%__

__ (e) 33.3%__

Plausible Answer Strategies

Use Nonsense Words ** **: __ __

__ Colio Wines in Ontario produce an interesting product classified as Vin de Curé in much the same way as Amarone is made in Veneto. The Italian word describing the drying process used to desiccate the grapes is:__

__ a) Ripasso b) Appasimento__

__ c) Raisinato d) Pommace__

__ e) Prunello__

Adding Uncertainty..

The term “fortified wine” means:

__ a) the bottle had been reinforced__

__ b) the wine has had tannins and acid __

__ added__

__ c) the wine has had sugar added__

__ d) the wine has had SO__ 2 __ added__

__ e) none of the foregoing__

Adding difficulty..

*Choose the ** *best __ answer.__ __ __

__ Rootham’s, a well-known local manufacturer of brandied fruit preserves sells their product through gourmet food shops and at craft fairs throughout Canada. Their recipes use genuine brandy and all natural ingredients. They are able to sell these legally because:__

(a) the alcohol used is produced by fermentation

(b) the product is made with ethyl alcohol

(c) the alcohol used is purchased from the LCBO

(d) the alcohol concentration is less than 1.2%

(e) the label says “made with genuine brandy”

Making Multiple Choice Questions Interesting…

__ A gracious, but parsimonious hostess wants to receive guests in the private dining room of your restaurant with a glass of champagne just to tantalise the taste buds and set the mood for the evening. __

**1.You suggest that if the wine is being poured at the bar, she should consider serving a Canadian sparkling wine that is very similar to Champagne in quality but considerably less expensive. Such a wine would be a: **

2. The cheapest sparkling wine would be obvious to sophisticated guests because they would know that in the best sparkling wines…

**3. If the gracious hostess insists on having “French” Champagne and you are left to rationalise costs, the most cost effective solution is to serve Champagne from.. **

**4. Occasionally, a patron asks you to do something that will distinguish their event from others. One client wants to obtain a large bottle of Champagne to celebrate a particular event. Which is the largest bottle of champagne on the list below? **

5. If someone wanted to serve a “glass of bubbly” during a reception, which of the following would be the most ** ina **pp ro p riate __ type of wine to select as the pre-prandial (before dinner) libation? __

Evaluating Multiple Choice Questions

Difficulty Factor = R/N

**R = # of times the correct answer is **

__ selected__

N = # of people taking the exam

Difficulty Factor

Evaluating Multiple Choice Questions

Good Discrimination

Reverse Discrimination

Discrimination Index

For the Correct Answer Option

__ Values ~ 0.5 give good discrimination__

__ < 0.2 consider dropping__

__ negative… drop the question__

For Incorrect Answer Options

__ Values should be negative.__

Knowledge or Successful Guessing?

Possibility of a “Random Pass”

**Depends on the number **

of answer options per question

and the number of questions!

Number of Questions

Percent Pass (≥50%) by Chance

from Brown, 2001

Adjustment for Guessing

Negative Marking..

Teaching With Multiple Choice Questions

Prepare lectures incorporating all material covered in the questions to be used

After each lecture, check coverage by reviewing questions

Provide students with multiple choice questions for self-evaluation at the end of readings

Teaching With Multiple Choice Examinations

__ Use stem questions to teach additional or ancillary terms not covered in the course:__

__ i.e. …. For post-prandial (after dinner) drinks you want to stock a supply of fortified wines and brandy. The term “fortified wine” means….__

Provide Students the Stem Questions Prior to the Exam!

Exam questions should cover all the course material being tested

Provides more detail than a course outline

Students focus their study effort on the topics they understand the least

Increases learning

Reduces stress

Thanks for your interest in my topic!

I look forward to your questions and ideas.

J.E. (Joe) Barth

School of Hospitality and Tourism Management

University of Guelph, Ontario CANADA