JDSE
Journal of Dental Sciences and Education

The Journal of Dental Sciences and Education deals with General Dentistry, Pediatric Dentistry, Restorative Dentistry, Orthodontics, Oral diagnosis and DentomaxilloFacial Radiology, Endodontics, Prosthetic Dentistry, Periodontology, Oral and Maxillofacial Surgery, Oral Implantology, Dental Education and other dentistry fields and accepts articles on these topics. Journal of Dental Science and Education publishes original research articles, review articles, case reports, editorial commentaries, letters to the editor, educational articles, and conference/meeting announcements.

eISSN: 2980-1389
Creative Commons CC BY-NC-ND 4.0 International License.
EndNote Style

Online Manuscript System

Index
Original Article
Comparative evaluation of AI-based chatbots in answering prosthodontic questions from the Turkish Dental Specialty Exam (DUS)
Abstract
Aims: This study compared the performance of the ChatGPT, Gemini, Copilot, and DeepSeek large language model–based chatbots in answering prosthodontic questions from the Turkish Dental Specialty Examination (DUS).
Methods: A total of 126 prosthodontic questions from DUS examinations conducted between 2012 and 2021 were included in the study after excluding visually based and canceled questions. The questions were classified as knowledge-based and case-based. Additionally, they were categorized according to topic areas as fixed prosthodontics, removable prosthodontics and implant-supported prosthodontics. The data were analyzed using the generalized estimating equations model, which accounts for repeated measurements, and differences between models were evaluated using Bonferroni-corrected post hoc tests.
Results: All chatbot models included in this study achieved an overall accuracy rate above 80% (ChatGPT-4o: 85.71%; Copilot: 83.33%; Gemini 3 Flash: 81.75%; DeepSeek-V3.2: 80.16%). No statistically significant difference was observed among the models in terms of overall performance (p = 0.619). All models demonstrated significantly higher performance on knowledge-based questions than on case-based questions (p < 0.05). Topic-based analysis showed numerically lower accuracy for implant-supported prosthodontic questions than for fixed and removable prosthodontic questions. However, this finding should be interpreted cautiously because the implant-supported subgroup included only four questions. No significant relationship was found between examination years and the accuracy rates of the models (?² = 7.1; p = 0.62).
Conclusion: The examined artificial intelligence models demonstrated a high level of accuracy in answering prosthodontic specialty questions. These systems have the potential to serve as supportive tools in dental education and examination preparation processes. However, owing to their limitations in questions requiring clinical reasoning, particularly case-based questions, their use should be approached cautiously and critically.

Keywords : Artificial intelligence, large language models, prosthodontics, DUS, dental education

Download Article


1. Hoch CC, Wollenberg B, Lüers JC, et al. ChatGPT’s quiz skills in different otolaryngology subspecialties: an analysis of 2576 single-choice and multiple-choice board certification preparation questions. Eur Arch Otorhinolaryngol. 2023;280(9):4271-4278. doi:10.1007/s00405-023-08051-4
2. Sismanoglu S, Capan BS. Performance of Artificial Intelligence on Turkish Dental Speciali-zation Exam: can ChatGPT-4.0 and Gemini Advanced achieve comparable results to hu-mans? BMC Med Educ. 2025;25(1):214. doi:10.1186/s12909-024-06389-9
3. Fogel AL, Kvedar JC. Artificial Intelligence powers digital medicine. NPJ Digit Med. 2018;1:5. doi:10.1038/s41746-017-0012-2
4. Cicciù M, Fiorillo L, D’Amico C, et al. 3D digital impression systems compared with tradi-tional techniques in dentistry: a recent data systematic review. Materials. 2020;13(8):1982. doi:10.3390/ma13081982
5. Sezer B, Aydoğdu T. Performance of advanced Artificial Intelligence models in pulp therapy for immature permanent teeth: a comparison of ChatGPT-4 Omni, DeepSeek, and Gemini Advanced in accuracy, completeness, response time, and readability. J Endod. 2025;51(11):1675-1684. doi:10.1016/j.joen.2025.08.011
6. Iannantuono GM, Bracken-Clarke D, Floudas CS, Roselli M, Gulley JL, Karzai F. Applica-tions of large language models in cancer care: current evidence and future perspectives. Front Oncol. 2023;13:1268915. doi:10. 3389/fonc.2023.1268915
7. Zhang P, Kamel Boulos MN. Generative AI in medicine and healthcare: promises, opportu-nities and challenges. Future Internet. 2023;15(9):286. doi:10.3390/fi15090286
8. Sallam M. ChatGPT utility in healthcare education, research, and practice: systematic re-view on the promising perspectives and valid concerns. Healthcare. 2023;11(22):2955. doi:10.3390/healthcare11222955
9. Brown T, Mann B, Ryder N, et al. Language models are few-shot learners. Adv Neural Inf Process Syst. 2020;33:1877-1901. doi:10.48550/arXiv.2005.14165
10. Eraslan R, Ayata M, Yagci F, Albayrak H. Exploring the potential of Artificial Intelligence chatbots in prosthodontics education. BMC Med Educ. 2025;25(1):321. doi:10.1186/s12909-025-06849-w
11. Dave M, Patel N. Artificial Intelligence in healthcare and education. Br Dent J. 2023;234(10):761-764. doi:10.1038/s41415-023-5845-2
12. Tartuk BK, Altıntaş E. Evaluation of accuracy, clinical reliability and readability of LLM-based chatbot responses in prosthetic dentistry FAQs. Anatolian Curr Med J. 2026;8(3):513-523. doi:10.38053/acmj.1905988
13. Çulhaoğlu AK, Kılıçarslan MA, Denız KZ. Diş Hekimliğinde Uzmanlik Sınavının farklı eğitim seviyelerdeki algı ve tercih durumlarının değerlendirilmesi. Ata Diş Hek Fak Derg. 2021;31(3):420-426. doi:10.17567/ataunidfd.911839
14. Tosun B, Yilmaz ZS. Comparison of Artificial Intelligence systems in answering prostho-dontics questions from the dental specialty exam in Turkey. J Dent Sci. 2025;20(3):1454-1459. doi:10.1016/j.jds.2025.01.025
15. Venkatesan S, Krishnamoorthi D, Raju R, Mohan J, Thomas PA, Rubasree B. Evidence-based prosthodontics. J Pharm Bioallied Sci. 2022;14(Suppl 1):S50-S59. doi:10.4103/jpbs.jpbs_149_22
16. Dundar Sari MB, Sezer B. Comparative performance evaluation of ChatGPT-4 Omni and Gemini Advanced in the Turkish Dentistry Specialization Exam. BMC Med Educ. 2026;26(1):251. doi:10.1186/s12909-026-08621-0
17. Alowais SA, Alghamdi SS, Alsuhebany N, et al. Revolutionizing healthcare: the role of Arti-ficial Intelligence in clinical practice. BMC Med Educ. 2023;23(1):689. doi:10.1186/s12909-023-04698-z
18. Erdal SG, Güdül A, Köroğlu A. The effectiveness of large language models in dental special-ty questions: a comparative study in the field of prosthodontics. BMC Med Educ. 2026;26(1):455. doi:10.1186/s12909-026-08808-5
19. Yilmaz BE, Gokkurt Yilmaz BN, Ozbey F. Artificial Intelligence performance in answering multiple-choice oral pathology questions: a comparative analysis. BMC Oral Health. 2025;25(1):573. doi:10.1186/s12903-025-05926-2
20. Kung TH, Cheatham M, Medenilla A, et al. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023;2(2):e0000198. doi:10.1371/journal.pdig.0000198
21. Kasagga A, Sapkota A, Changaramkumarath G, et al. Performance of ChatGPT and large language models on medical licensing exams worldwide: a systematic review and network meta-analysis with meta-regression. Cureus. 2025;17(10):e94300. doi:10.7759/cureus.94300
22. Temiz M, Güzel C. Assessing the performance of ChatGPT on dentistry specialization exam questions: a comparative study with DUS examinees. Medical Records. 2025;7(1):162-166. doi:10.37990/medr.1567242
23. Ekici Ö. Comparative evaluation of four large language models in Turkish Dentistry Spe-cialization Exam. Selcuk Dent J. 2025;12(4):6-10. doi:10.15311/selcukdentj.1674113
24. Haberal M, Hançerlioğulları D. Can Artificial Intelligence chatbots think like dentists? A comparative analysis based on dental specialty examination questions in restorative dentis-try. BMC Oral Health. 2026;26(1):231. doi:10.1186/s12903-025-07612-9
25. Kim J, Podlasek A, Shidara K, Liu F, Alaa A, Bernardo D. Limitations of large language models in clinical problem-solving arising from inflexible reasoning. Sci Rep. 2025;15(1):39426. doi:10.1038/s41598-025-22940-0
26. Karvekar S, Anand V, Singh D, Chaudhary VK, Fatima S, Mathar MI. Digital dentistry and Artificial Intelligence: a systematic review on innovations in diagnosis, treatment planning, and prosthodontics. Cureus. 2026;18(2):e104422. doi:10.7759/cureus.104422
27. Alfaraj A, Limones Á, Ahmad S, et al. Harnessing AI in prosthodontics and implant dentis-try: an umbrella review of systematic evidence. J Prosthodont. 2026;35(2):127-142. doi:10.1111/jopr.70091
28. Revilla-León M, Gómez-Polo M, Vyas S, et al. Artificial Intelligence applications in implant dentistry: a systematic review. J Prosthet Dent. 2023;129(2):293-300. doi:10.1016/j.prosdent.2021.05.008
29. Turan Gökduman C, Arılı Öztürk E, Aktaş Ş, Çanakçi BC. Comparison of chatbots’ accuracy in endodontics questions in dentistry specialization exam in Turkiye: ChatGPT-4o, Gemini Advanced, Copilot, and Claude. BMC Oral Health. 2025;26(1):28. doi:10.1186/s12903-025-07346-8
30. Tassoker M. ChatGPT-4 Omni’s superiority in answering multiple-choice oral radiology questions. BMC Oral Health. 2025;25(1):173. doi:10.1186/s12903-025-05554-w
31. Çetin SG, Karadağ GG. Comparative evaluation of Artificial Intelligence models in answer-ing pediatric dentistry questions of the Turkish Dental Specialization Exam (DUS). Dicle Dent J. 2025;26(2):31-37.
32. Kong M, Fok EHW, Yiu CKY. A Scoping review of large language models in dental educa-tion: applications, challenges, and prospects. Int Dent J. 2025;75(6):103854. doi:10.1016/j.identj.2025.103854
33. Roustan D, Bastardot F. The clinicians’ guide to large language models: a general perspec-tive with a focus on hallucinations. Interact J Med Res. 2025;14:e59823. doi:10.2196/59823
34. Aşık A, Kuru E. Analysis of ChatGPT’s answers to pedodontics questions asked in the den-tistry specialization training entrance exam: cross-sectional study. Turkiye Klinikleri J Den-tal Sci. 2025;31(3):401-406. doi:10.5336/dentalsci.2024-107488
35. Mavrych V, Ganguly P, Bolgova O. Using large language models (ChatGPT, Copilot, PaLM, Bard, and Gemini) in gross anatomy course: comparative analysis. Clin Anat. 2025;38(2):200-210. doi:10.1002/ca. 24244
Article available in :
Volume 4, Issue 3, 2026
Page : 84-90
https://doi.org/10.51271/JDSE-0082
Article Information
Received : 01 Haz 2026
Accepted : 30 Tem 2026
Published : 20 Ağu 2026
Corresponding Author :

Bülent Kadir Tartuk: Department of Prosthodontics, Faculty of Dentistry, Bingöl University, Bingöl, Turkiye

[email protected]
Citation : Tartuk BK. Comparative evaluation of AI-based chatbots in answering prosthodontic questions from the Turkish Dental Specialty Exam (DUS). J Dent Sci Educ. 2026;4(3):84-90.

Read: 1
Download : 0
Cited: 0
_Footer