Movie-ing the EEG: a Hybrid CNN-Transformer-based Framework for decoding EEG signals for BCI applications

This paper introduces a hybrid deep learning approach for processing electroencephalographic (EEG) signals to track their dynamics over time in the spatial-spectral domain. This method is particularly valuable whenever the temporal evolution of the brain process under analysis is relevant to classification. A hybrid deep learning model, herein referred to as EEGConViT, processes spatial-temporal stream of data (EEG movies). Specifically, EEGConViT consists of a custom CNN that encodes each frame into a feature vector, which is then augmented with temporal position embeddings with a Transformer model able to capture sequential dependencies. In this work, the application of motor EEG signals was investigated with the proposed EEGConViT. In particular, EEG signals preceding motor execution were processed to assess the ability of the model to predict the upcoming sub-movement of the upper limb. The model was evaluated using a collection of EEG signals from 14 subjects, derived from a publicly available repository. Using a leave-one-subject-out strategy, the model was trained on data from 13 subjects and fine-tuned on the remaining one (cross-subject training and subsequent calibration over the single subject). Results demonstrate that our approach outperforms comparable models in the literature while significantly reducing training time, an essential factor in medical applications, where both classification performance and rapid calibration are critical.