Manual for ESTRO Teachers

UNION EUROPÉENNE DES MÉDEC INS SPÉC IAL I STES

EUROPEAN UNION OF MEDI CAL SPEC IAL I STS

Association internationale sans but lucratif – International non-profit organisation

VENUE DE LA

OURONNE

, 20 3

T +32 2 649 51 64

BE- 1050 BRUSSELS

F +32 2 640 37 30

the cognitive domain). Because of this, it is often argued that the complexity of day-to-day clinical

medicine cannot be reflected in such an apparently “simple” assessment tool. Nevertheless,

although MCQs are typically better at detecting recall of facts rather than understanding of concepts,

it is possible to write conceptually based items to test higher levels of understanding such as

evaluation

and

synthesis

through careful writing of creative questions. Therefore, the criticisms

voiced against the concept of MCQs should be directed towards the flaws in the construction of the

items rather than to their inherent weakness.

It is common knowledge that the “multiple choice question” is the most common type of written test

item used in undergraduate, graduate and post-graduate medical education. The principles of writing

effective MCQs are well documented in test-item construction manuals, educational measurement

textbooks and the research literature. Despite this a study from the National Board of Medical

Examiners showed that violations of the most basic item-writing principles are very common in

medical education examinations.

There is no doubt that it is harder to write a concept-based MCQ rather than a knowledge-based one

and it is also harder to write a good MCQ than a bad one.

One approach to overcome flaws in construction is to write questions where the participant is asked

to make a (multiple-choice) decision whilst providing some evidence/data upon which to base the

decision. The MCQ options presented should all represent plausible decision options, the correct

selection of which, requires the interpretation of the evidence and the application of appropriate

decision making methodologies. In this way the questions seek to test clinical reasoning and

judgement rather than logic or an ability to pass exams.

After reviewing the extensive literature concerning the best and most appropriate methods to assess

factual/conceptual knowledge it is clear that MCQs have become accepted as the “least flawed”

method of the current forms of assessment, especially compared to the “traditional” oral

examinations. The highly subjective nature of the multiple components of oral examinations leaves

this method of assessment potentially wide open to legal challenges. In contrast to oral

examinations, MCQs are by their nature objective, easy to grade, efficient in time and allow some

degree of comparison between learners, information which is highly regarded and most constructive

both in a “formative” as well as a “summative” examination. In summary, although MCQs have

obvious limitations, in terms of objectivity and efficiency, because alternative methods are

unsatisfactory, they will in future provide one component of the assessment of doctors.