Personalized Split Federated Learning With Early Exit: Pretraining and Online Learning Against Label Shifts

초록

Advancements in artificial intelligence (AI) have enabled Internet of Things (IoT) devices to offer intelligent services, improving system adaptability and scalability. split federated learning (SFL) has emerged as a promising approach for privacy-sensitive and resource-constrained IoT devices, addressing computational and privacy challenges. In the SFL framework, IoT devices serve as clients and do not share raw client data with the server. Instead, they offload computationally intensive tasks to the server. However, an SFL-based IoT system faces three key challenges. First, it struggles to personalize client models when client data distributions are heterogeneous. Second, it encounters a tradeoff between communication overhead and data privacy. Third, it suffers from severe performance degradation when label distributions shift after deploying the pretrained model. To address these issues, we propose a novel early exit SFL framework consisting of a pretraining phase and an online learning phase. In the pretraining phase, we introduce a personalized SFL training method to tailor each client model to its data distribution and a surrogate target generation method to train the server's large model. In the online learning phase, client models are updated to handle label distribution shifts and maintain performance by leveraging the server's large model as a teacher through knowledge distillation. For the real-time inference, early exit is available by passing through only the client's model. Extensive simulations demonstrate that the proposed framework achieves superior accuracy and lower communication costs compared to state-of-the-art methods. Notably, our method outperforms existing methods with average improvements of 18.49% in pretraining settings and 30.96% under diverse label distribution shift scenarios.

키워드