Multi-teacher Based Knowledge Distillation for Retinal Vessel Segmentation

Abstract

Accurate segmentation of retinal vessels is crucial for the early diagnosis and management of various ocular diseases. Existing methods often struggle to segment thin vessels, leading to missed diagnoses and inaccurate treatment plans. This study proposes a novel Multi-Teacher Based Knowledge Distillation (MTKD) method for Retinal Vessel Segmenta tion (RVS) to address this challenge. Our approach utilizes the expertise of multiple teacher networks, each specialized in learning different vessel characteristics. Specifically, we train three distinct teacher networks: one on the original ground truth, one on a modified ground truth highlighting thin vessels, and another on a modified ground truth emphasizing thick vessels. The student network is then trained to minimize the knowledge discrepancy between its predictions and the soft predictions of all three teachers. By incorporating knowledge from these specialized teachers, the student network effectively learns to segment both thin and thick vessels with improved accuracy. We evaluate our method on two retinal fundus image datasets and two angiography datasets, demonstrating highly competitive performance compared to state-of-the-art methods. The proposed method improves the baseline U-Net model by up to 8.44 points in F1 and 10.42 points in IOU. Additionally, we introduce a penalization technique to the student model’s loss function, further enhancing segmentation performance. Comprehensive ablation studies validate the effectiveness of the multi-teacher approach, the choice of loss functions, and the impact of model complexity. Our f indings suggest that MTKD offers a promising approach for enhancing the robustness and accuracy of RVS. All source code, datasets, and results are made publicly available to support reproducibility and further research.