EVALUATING THE DIFFICULTY AND DISCRIMINATION OF THE FINAL EXAMINATION FOR 6TH-YEAR MEDICAL STUDENTS - NGUYEN TAT THANH UNIVERSITY FROM CTT TO MIRT APPROACH

Anh Duy Vu1, , Minh Nguyet Le1, Van Tri Nguyen1, Ngoc Sinh Tran1, Huu Tung Trinh1, Quang Thanh Le1, Van Quang Hoang1, Trong Anh Do1, Thi Kim Nhung Le1
1 Nguyen Tat Thanh University

Main Article Content

Abstract

Background: In addition to Classical Test Theory, Item Response Theory, particularly the two-parameter logistic model and the multidimensional model—provides deeper insights into item difficulty, discrimination, and the latent structure of examinations. Objectives: To evaluate the psychometric properties of the final examination for medical graduates at Nguyen Tat Thanh University by combining analyses from three methods. Materials and methods: A cross-sectional descriptive study was conducted on 118 sixth-year medical students, using a 120-item multiple-choice final examination. Classical test theory analysis and the Item Response Theory model were used to estimate item difficulty and discrimination parameters. A simple-structure multidimensional model was then employed to assess item characteristics within majors. Results: The test demonstrated acceptable reliability (Cronbach’s alpha = 0.769). Based on classical test theory, 15 items (12.8%) were difficult, and 40 items (34.2%) were easy. In terms of discrimination, 80 out of 120 items showed poor discriminative ability. Under the Item Response Theory analysis, 32 items achieved at least moderate discrimination (a > 0.65). A total of 57 items were classified as easy (b < –2), while only 24 were considered difficult (b > 1). In the simple-structure multidimensional model, the Internal Medicine domain performed best with 12 well-discriminating items, whereas the Surgery domain had 16 poorly discriminating items. Conclusion: The integration of those methods enables a comprehensive evaluation of the psychometric properties of examinations, thereby providing guidance for improving test quality. 

Article Details

References

1. Bùi Anh Tú, Võ Đăng Khoa, Nguyễn Anh Vũ. Đánh giá đề thi MCQ đầu vào và tương quan độ khó, độ phân cách các câu. Tạp chí Y học Thành phố Hồ Chí Minh. 2024. 27(5), 38-47, doi: 10.32895/hcjm.m.2024.05.05.
2. Phạm Dương Uyển Bình, Trần Thị Diệu. Mối quan hệ giữa độ khó năng lực và độ phân cách của câu hỏi trắc nghiệm sinh lý học trong các đề thi tuyển sinh sau đại học dành cho đối tượng chuyên khoa cấp I từ năm 2018-2022 tại Đại học Y Dược Thành Phố Hồ Chí Minh. Tạp chí Y học Thành phố Hồ Chí Minh. 2024. 27 (1), 170-176, doi: 10.32895/hcjm.m.2024.01.24.
3. T. Rusch, P. Lowry, P. Mair, and H. Treiblmaier. Breaking Free from the Limitations of Classical Test Theory: Developing and Measuring Information Systems Scales Using Item Response Theory. Information & Management. 2017. 54 (2),189-203, doi: 10.1016/j.im.2016.06.005.
4. Nathan Thompson. Classical Test Theory vs Item Response Theory. 2023. https://assess.com/classical-test-theory-vs-item-response-theory.
5. Pimentel J.L., Villaruz M.L.A. Comparison of item difficulty estimates in a basic statistics test using ltm and CTT software packages in R. International Journal of Advanced Computer Science and Applications. 2020. 11(3), 367-372, doi: 10.14569/IJACSA.2020.0110346.
6. De Ayala R.J. The Theory and Practice of Item Response Theory. Guilford Press. 2022.
7. Paek I., Cole K. Using R for Item Response Theory Model Applications. Routledge. 2020.
8. Tavakol M, Dennick R. Making sense of Cronbach's alpha. Int J Med Educ. 2011. 2, 53-55, doi: 10.5116/ijme.4dfb.8dfd.