Exploring a Transformer Approach for Pigment Signs Segmentation in Fundus Images

Over the past couple of years, Transformers became increas- ingly popular within the deep learning community. Initially designed for Natural Language Processing tasks, Transformers were then tailored to fit to the Image Analysis field. The self-attention mechanism behind Transformers immediately appeared a promising, although computation- ally expensive, learning approach. However, Transformers do not adapt as well to tasks involving large images or small datasets. This propelled the exploration of hybrid CNN-Transformer models, which seemed to overcome those limitations, thus sparkling an increasing interest also in the field of medical imaging. Here, a hybrid approach is investigated for Pigment Signs (PS) segmentation in Fundus Images of patients suffering from Retinitis Pigmentosa, an eye disorder eventually leading to com- plete blindness. PS segmentation is a challenging task due to the high variability of their size, shape and colors and to the difficulty to dis- tinguish between PS and blood vessels, which often overlap and display similar colors. To address those issues, we use the Group Transformer U-Net, a hybrid CNN-Transformer. We investigate the effects, on the learning process, of using different losses and choosing an appropriate parameter tuning. We compare the obtained performances with the clas- sical U-Net architecture. Interestingly, although the results show margins for a consistent improvement, they do not suggest a clear superiority of the hybrid architecture. This evidence raises several questions, that we address here but also deserve to be further investigated, on how and when Transformers are really the best choice to address medical imaging tasks.