Background: ChatGPT is an open-source natural language processing software that replies to users' queries. We conducted a cross-sectional study to assess people living with Multiple Sclerosis' (PwMS) preferences, satisfaction, and empathy toward two alternate responses to four frequently-asked questions, one authored by a group of neurologists, the other by ChatGPT. Methods: An online form was sent through digital communication platforms. PwMS were blind to the author of each response and were asked to express their preference for each alternate response to the four questions. The overall satisfaction was assessed using a Likert scale (1-5); the Consultation and Relational Empathy scale was employed to assess perceived empathy. Results: We included 1133 PwMS (age, 45.26 ± 11.50 years; females, 68.49%). ChatGPT's responses showed significantly higher empathy scores (Coeff = 1.38; 95% CI = 0.65, 2.11; p > z < 0.01), when compared with neurologists' responses. No association was found between ChatGPT' responses and mean satisfaction (Coeff = 0.03; 95% CI = - 0.01, 0.07; p = 0.157). College graduate, when compared with high school education responder, had significantly lower likelihood to prefer ChatGPT response (IRR = 0.87; 95% CI = 0.79, 0.95; p < 0.01). Conclusions: ChatGPT-authored responses provided higher empathy than neurologists. Although AI holds potential, physicians should prepare to interact with increasingly digitized patients and guide them on responsible AI use. Future development should consider tailoring AIs' responses to individual characteristics. Within the progressive digitalization of the population, ChatGPT could emerge as a helpful support in healthcare management rather than an alternative.

ChatGPT vs. neurologists: a cross-sectional study investigating preference, satisfaction ratings and perceived empathy in responses among people living with multiple sclerosis / Maida, Elisabetta; Moccia, Marcello; Palladino, Raffaele; Borriello, Giovanna; Affinito, Giuseppina; Clerico, Marinella; Repice Anna, Maria; Di Sapio, Alessia; Iodice, Rosa; Spiezia, ANTONIO LUCA; Sparaco, Maddalena; Miele, Giuseppina; Bile, Floriana; Scandurra, Cristiano; Ferraro, Diana; Stromillo Maria, Laura; Docimo, Renato; De Martino, Antonio; Mancinelli, Luca; Abbadessa, Gianmarco; Smolik, Krzysztof; Lorusso, Lorenzo; Leone, Maurizio; Leveraro, Elisa; Lauro, Francesca; Trojsi, Francesca; Streito Lidia, Mislin; Gabriele, Francesca; Marinelli, Fabiana; Ianniello, Antonio; De Santis, Federica; Foschi, Matteo; De Stefano, Nicola; BRESCIA MORRA, Vincenzo; Bisecco, Alvino; Coghe, Giancarlo; Cocco, Eleonora; Romoli, Michele; Corea, Francesco; Leocani, ANNUNZIATA MARIA LETIZIA; Frau, Jessica; Sacco, Simona; Inglese, Matilde; Carotenuto, Antonio; Lanzillo, Roberta; Padovani, Alessandro; Triassi, Maria; Bonavita, Simona; Lavorgna, Luigi. - In: JOURNAL OF NEUROLOGY. - ISSN 0340-5354. - (2024). [10.1007/s00415-024-12328-x]

ChatGPT vs. neurologists: a cross-sectional study investigating preference, satisfaction ratings and perceived empathy in responses among people living with multiple sclerosis

Moccia Marcello;Palladino Raffaele;Borriello Giovanna;Affinito Giuseppina;Iodice Rosa;Spiezia Antonio Luca;Scandurra Cristiano;Lauro Francesca;Gabriele Francesca;Ianniello Antonio;Foschi Matteo;Morra Vincenzo Brescia;Bisecco Alvino;Leocani Letizia;Carotenuto Antonio;Lanzillo Roberta;Triassi Maria;
2024

Abstract

Background: ChatGPT is an open-source natural language processing software that replies to users' queries. We conducted a cross-sectional study to assess people living with Multiple Sclerosis' (PwMS) preferences, satisfaction, and empathy toward two alternate responses to four frequently-asked questions, one authored by a group of neurologists, the other by ChatGPT. Methods: An online form was sent through digital communication platforms. PwMS were blind to the author of each response and were asked to express their preference for each alternate response to the four questions. The overall satisfaction was assessed using a Likert scale (1-5); the Consultation and Relational Empathy scale was employed to assess perceived empathy. Results: We included 1133 PwMS (age, 45.26 ± 11.50 years; females, 68.49%). ChatGPT's responses showed significantly higher empathy scores (Coeff = 1.38; 95% CI = 0.65, 2.11; p > z < 0.01), when compared with neurologists' responses. No association was found between ChatGPT' responses and mean satisfaction (Coeff = 0.03; 95% CI = - 0.01, 0.07; p = 0.157). College graduate, when compared with high school education responder, had significantly lower likelihood to prefer ChatGPT response (IRR = 0.87; 95% CI = 0.79, 0.95; p < 0.01). Conclusions: ChatGPT-authored responses provided higher empathy than neurologists. Although AI holds potential, physicians should prepare to interact with increasingly digitized patients and guide them on responsible AI use. Future development should consider tailoring AIs' responses to individual characteristics. Within the progressive digitalization of the population, ChatGPT could emerge as a helpful support in healthcare management rather than an alternative.
2024
ChatGPT vs. neurologists: a cross-sectional study investigating preference, satisfaction ratings and perceived empathy in responses among people living with multiple sclerosis / Maida, Elisabetta; Moccia, Marcello; Palladino, Raffaele; Borriello, Giovanna; Affinito, Giuseppina; Clerico, Marinella; Repice Anna, Maria; Di Sapio, Alessia; Iodice, Rosa; Spiezia, ANTONIO LUCA; Sparaco, Maddalena; Miele, Giuseppina; Bile, Floriana; Scandurra, Cristiano; Ferraro, Diana; Stromillo Maria, Laura; Docimo, Renato; De Martino, Antonio; Mancinelli, Luca; Abbadessa, Gianmarco; Smolik, Krzysztof; Lorusso, Lorenzo; Leone, Maurizio; Leveraro, Elisa; Lauro, Francesca; Trojsi, Francesca; Streito Lidia, Mislin; Gabriele, Francesca; Marinelli, Fabiana; Ianniello, Antonio; De Santis, Federica; Foschi, Matteo; De Stefano, Nicola; BRESCIA MORRA, Vincenzo; Bisecco, Alvino; Coghe, Giancarlo; Cocco, Eleonora; Romoli, Michele; Corea, Francesco; Leocani, ANNUNZIATA MARIA LETIZIA; Frau, Jessica; Sacco, Simona; Inglese, Matilde; Carotenuto, Antonio; Lanzillo, Roberta; Padovani, Alessandro; Triassi, Maria; Bonavita, Simona; Lavorgna, Luigi. - In: JOURNAL OF NEUROLOGY. - ISSN 0340-5354. - (2024). [10.1007/s00415-024-12328-x]
File in questo prodotto:
File Dimensione Formato  
Maida et al. (2024) - ChatGPT and SM.pdf

accesso aperto

Descrizione: Maida et al. (2024) - ChatGPT and SM
Tipologia: Documento in Post-print
Licenza: Dominio pubblico
Dimensione 671.77 kB
Formato Adobe PDF
671.77 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11588/957691
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact