Item Analysis of Pathology Assessment of 4 year MBBS at Rawalpindi Medical University Pakistan

Background: One of the ways to improve the standard of our medical education is to assess the quality of Multiple-Choice Questions (MCQs) imparted in our undergraduate assessments. Objectives: To determine the difficulty index, discrimination index, distractor effectiveness, and reliability of Multiple-Choice Questions (MCQs) of Pathology attempted by 4th-year MBBS students. Materials & Methods: A cross-sectional descriptive research was done to analyze Pathology MCQs based test that was attempted online by 4th-year MBBS students at Medical University during January 2021 due to COVID-19 pandemic. 112 papers of high performers and 112 papers of low performers were enrolled in this study through purposive sampling. Data were analyzed by using SPSS version 25.0 and Microsoft Excel 2007. Difficulty index, discrimination index, distractor effectiveness, reliability co-efficient of MCQs was computed. An independent sample t-test was used to determine the difference between the difficulty index and the discrimination index. P-value < 0.05 was considered as significant. Results: 60 % MCQs were labeled as easy while 40 % were moderately difficult. None of the items was very difficult. The mean difficulty index and discrimination index were 75.67 ± 12.9 and 0.27 ± 0.14, respectively. Of the total 30, MCQs detected as easy, about 50 % were poor discriminators while 50 % of the moderately difficult items were excellent discriminators. About 87.5 % of distractors were functional. Items had perfect reliability with KR-20 of 1.0. Statistically significant difference (P < 0.00) was observed between difficulty index and discrimination index on applying an independent sample t-test. Conclusion: MCQs had sufficiently acceptable quality. Critically appraising the results of item analysis may substantially enhance the standard


Introduction
Multiple Choice Questions (MCQs) are valued as a multifaceted tool for the evaluation of medical education across the globe due to their capacity to assess massive students on varied topics in a very short span of time [1]. Well-structured MCQs are endowed to adequately assess the higher-ordered thinking skills of the candidates [2]. They are also proven to be the abundantly used instrument for authentically assessing the learning outcomes [3].
It is imperative to have valid, reliable, and objective assessment tool to ensure adequate reflectivity of diverse achievement levels [4].
MCQs are frequently incorporated in undergraduate as well postgraduate exams of medical students for their thorough assessment [5]. They can appraise the higher-order cognitive domains instead of just judging the memorized knowledge [6]. Moreover, they are substantially remarkable in differentiating high achievers from low achievers [7].
Item analysis of MCQs is quite advantageous in determining the quality of questions in terms of difficulty index, discrimination index, and distractor effectiveness. In addition to lack of internal consistency among tests, items can also be ruled out by psychometric analysis [8].
Its prime objective is either to revise the multiple-choice question or replace them with some other adequate difficulty level and discrimination index question so that candidates' knowledge and competencies could be assessed in a true sense [9]. Incorporation of non-standardized options in MCQs will not only hampers recall of knowledge by the students but will also lead to guessing [10]. Good quality MCQs apart from promoting problem-solving and critical thinking skills among our students are also reflective of teachers' remarkable efforts in their designing [11]. Amendments in MCQs in accordance with their item analysis are truly profitable in their upgradation [12].
The present study is therefore planned to scrutinize the key attributes of multiple-choice questions of Pathology send up exam administered to 4th-year MBBS students at Rawalpindi Medical University during January 2021. The concerned stakeholders would really be benefited by the valuable suggestions to amplify the standard of questions.

Materials & methods
A cross-sectional descriptive study was carried out by a analyzing depending on the presence of 0, 1, 2, and 3 distractors respectively [15].
Reliability ranges from 0.00-1.00. KR-20 > 0.90 confirms homogeneity of items [16]. The mean and median of the scores achieved were also calculated. The difference between the difficulty index and the discrimination index was computed by means of an independent sample t-test. P-value < 0.05 was taken as significant.

Results
Of the total 50 MCQs assessed in our study, about 60 % of items were very easy and 40 % were moderately difficult. However, 62 % of items were determined to have good discriminating ability as depicted below in Table 1 and Table 2.     Table 3.

Conclusion & recommendations
Item analysis of multiple-choice questions is a constructive approach towards the provision of an insight into their quality deemed essential for accurately assessing the students' knowledge. Ample time should be spent on capacity building and designing standard MCQs.
Structuring remarkable MCQs with apt functional distractors are necessitated to ensure objectivity and reliability of assessment.