Manual for ESTRO Teachers

UNION EUROPÉENNE DES MÉDEC INS SPÉC IAL I STES

EUROPEAN UNION OF MEDI CAL SPEC IAL I STS

Association internationale sans but lucratif – International non-profit organisation

VENUE DE LA

OURONNE

, 20 8

T +32 2 649 51 64

BE- 1050 BRUSSELS

F +32 2 640 37 30

Responses:

1) Each multiple choice item should have five mutually exclusive responses

2) The best answer should be the one agreed upon by the experts; however, the other four

distractors should also seem plausible to the candidates who have partial, incomplete or

inappropriate knowledge.

a) There should be an evidence base for determining which of the answers are correct and

which are incorrect.

b) The location of that evidence - whether in international guidelines, textbooks or the

scientific literature - should be available to the question writer, the question-writing group

and to the candidates themselves.

c) The evidence must be cited within the curriculum that is being assessed.

3) The position of the “correct” response should vary from the A, B, C, D and E positions

a) Research shows that the B and C positions are overused.

b) “Testwise” candidates may key in on this fact and choose the B or C positions to increase

their chance of getting the answer right when forced to guess.

c) Keep correct answers in random positions (avoid any pattern formation that could

potentially be detected by candidates)

d) If necessary after the test is written, reorder the sequence where the correct answers are

placed

e) Another method is to have all responses to all questions listed alphabetically. However,

care must be taken with this approach as this can alter the correct response to certain

types of questions.

4) The “distractors” may be considered logical misconceptions of the best answer.

5) The grammatical structure of all the responses should be a logical conclusion to the question (or

statement) presented in the stem.

a) Because the author of a question tends to pay more attention to the correct option than

the distractors, grammatical errors (cueing the answer) are more likely to occur in the

distractors.

6) Repetitive language within the responses should be avoided.

a) Words which are repeated in every response may be placed in the stem so that the

candidate has less to read and is less likely to be confused by the structure.

7) The responses should be parallel in structure or category of information (e.g. all related to

“treatments”, or “diagnoses”, etc).