Harnessing Multi-modality and Expert Knowledge for Adverse Events Prediction in Clinical Notes

Recent advancements in machine learning and deep learning techniques have revolutionized the field of adverse event prediction, which plays a vital role in healthcare by enabling early identification and intervention for high-risk patients. Traditionally, researchers have relied on structured data, including demographic information, vital signs, laboratory results, and medication records. However, the widespread adoption of electronic health records (EHRs) has introduced a substantial amount of unstructured information in the form of clinical notes, which have been largely underutilized. Natural Language Processing (NLP) techniques have emerged as a powerful tool for extracting valuable insights from these clinical notes and incorporating them into machine learning frameworks. Additionally, multimodal machine learning, which integrates structured and unstructured data, has gained considerable attention to enhance the accuracy of adverse event prediction. This research focuses on the application of multimodal machine learning for predicting adverse events such as atrial fibrillation, heart failure, and ischemic myocardial infarction. The study aims to compare the performance of a Machine Learning specialist without domain knowledge would obtain with an approach guided by physicians, that includes an information retrieval step using unstructured clinical notes. The analysis is carried out using a dataset provided by the Hospital of Naples Federico II. The results not only shed light on the importance of leveraging different aspects of a patient’s medical history and extracting information from unstructured notes but also highlight the added value of domain expertise.