Analysis of multiple choice questions (MCQ): important part of assessment of medical students
Abstract
Background: Assessment influences students learning process that is why analysis of assessment allows us to conduct it properly and accurately. Tarrant M et al. and few others had done similar studies in past. Further studies are required to support the same and continuing awareness among teachers about better assessment of students.
Methods: The observational, non-interventional and prospective study was carried out to analyse 100 MCQs used for assessment of 2nd MBBS students. All MCQs were having single stem with four options, one being correct and other three distractors. Each MCQ was analyzed with three tools that were Difficulty Index (DIF I), Discrimination Index (DI) and Distracter Efficiency (DE). Chi square test was used for statistical analysis.
Results: Total 74 out of 100 MCQs (74%) were recommended (30-70%) according to DIF I. According to DI, total 34 out of 100 MCQs were good (0.25-0.35) and 30 MCQs were excellent (>0.35). There were 18 non-functional (6%) distractors out of total 300. In none of the MCQs, all three were poor distractor. Association between difficulty index and discrimination index was statistically significant according chi-square test. (p value< 0.05)
Conclusions: Properly constructed MCQs, according to these analysis tools, are best for student’s assessment.
Downloads
References
2. Scouller K. The influence of assessment method on students’ learning approaches: multiple-choice question examination versus assignment essay. Higher Educ. 1998; 35:453–72.
3. Trigwell K, Prosser M. Improving the quality of student learning: the influence of learning context and student approaches to learning on learning outcomes. Higher Educ. 1991; 22:251–66.
4. Biggs, J. Teaching for Quality Learning at University. The Society for Research into Higher Education & Open University Press, Philadelphia, 2003.
5. Reid WA, Duvall E, Evans P. Relationship between assessment results and approaches to learning and studying in Year Two medical students. Med Educ. 2007 Aug;41(8):754-62. [PubMed]
6. Abdel-Hameed AA, Al-Faris EA, Alorainy IA, Al-Rukban MO. The criteria and analysis of good multiple choice questions in a health professional setting. Saudi Med J. 2005 Oct;26(10):1505-10. [PubMed]
7. Case S, Swanson D. Constructing written test questions for the basic and clinical sciences. 3rd ed. Philadelphia: National Board of Medical Examiners, 2003.
8. Tarrant M, Ware J. A framework for improving the quality of multiple-choice assessments. Nurse Educ. 2012 May-Jun;37(3):98-104. doi: 10.1097/NNE.0b013e31825041d0. [PubMed]
9. Tarrant M, Ware J, Mohammed AM. An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis. BMC Med Educ. 2009 Jul 7;9:40. doi: 10.1186/1472-6920-9-40. [PubMed]
10. Hingorjo MR, Jaleel F. Analysis of one-best MCQs: the difficulty index, discrimination index and distractor efficiency. J Pak Med Assoc. 2012 Feb;62(2):142-7. [PubMed]
11. Singh T, Gupta P, Singh D. Principles of Medical Education. 3rd ed. New Delhi: Jaypee Brothers Medical Publishers (P) Ltd. Test and item analysis. 2009; 70–7.
12. Cizek GJ, O’Day DM: Further investigations of nonfunctioning options in multiple choice test items. Educ psycho meas. 1994, 54(4):861-72.
13. Entwistle N. Styles of Learning and Teaching. London: David Fulton Publishers. 1997; 77–107.
14. Downing SM: Assessment of knowledge with written test forms. In International handbook of research in medical education Volume II. Edited by: Norman GR, Van der Vleuten C, Newble DI. Dorcrecht: Kluwer Academic Publishers. 2002; 647-72.
15. McCoubrie P. Improving the fairness of multiple-choice questions: a literature review. Med Teach. 2004 Dec;26(8):709-12. [PubMed]
16. Schuwirth LW, van der Vleuten CP. Different written assessment methods: what can be said about their strengths and weaknesses? Med Educ. 2004 Sep;38(9):974-9. [PubMed]
17. Farley JK. The multiple-choice test: writing the questions. Nurse Educ. 1989 Nov-Dec;14(6):10-2, 39. [PubMed]
18. Haladyna TM, Downing SM: Validity of taxonomy of multiple choice item-writing rules. ApplMeasEduc. 1989; 2(1):51-78. [PubMed]
19. Sim SM, Rasiah RI. Relationship between item difficulty and discrimination indices in true/false-type multiple choice questions of a para-clinical multidisciplinary paper. Ann Acad Med Singapore. 2006 Feb;35(2):67-71.
20. Haladyna TM, Downing SM: How many options is enough for a multiple-choice test item? Educ Psychol Meas. 1993; 53(4): 999-1010.
21. Crehan KD, Haladyna TM, Brewer BW: Use of an inclusive option and the optimal number of options for multiple-choice items. Educ Psychol Meas. 1993; 53(1):241-7.
22. Rodriguez MC: Three options are optimal for multiple-choice items: A meta-analysis of 80 years of research. Educ Meas Issues Pract. 2005; 24(2):3-13.
23. Pellegrino J, Chudowsky N, Glaser R, editors. Knowing What Students Know: The Science and Design of Educational Assessment. Washington, DC: National Academic Press 2001.
OAI - Open Archives Initiative
