Quality and Readability of ChatGPT Performance on Parkinson's Disease

Background: Chat Generative Pre-trained Transformer (ChatGPT) is increasingly used for medical education and healthcare counseling. Objective: To evaluate the reliability, quality, usefulness, and readability of ChatGPT-4o responses to professional medical quizzes and patient frequently asked questions (FAQs) about Parkinson's disease (PD). Methods: Twenty-two quizzes were put into ChatGPT-4o and also given to two neurologists. ChatGPT answers to 20 FAQs were analyzed by three neurologists using the modified Discern score, the Global Quality score, and the Usefulness score. Text readability was assessed using the Flesch–Kincaid Reading Grade Level, the Flesch Reading Ease Score, and the Simple Measure of Gobbledygook. Results: The agreement between the neurologists and ChatGPT was fair. The answers to the FAQs were of good quality, but unreliable and difficult to read. Conclusions: ChatGPT may be an additional valid tool for professional medical education. The quality of the answers to the FAQs is good, but the text is aimed primarily at highly educated individuals. The readability and reliability of the output would likely improve by providing the chatbot with specific prompts.