Abstract
Federated Learning enables data owners to train an artificial intelligence model collaboratively while keeping all the training data locally, reducing the possibility of personal data breaches. However, the heterogeneity of local resources and dynamic characteristics of federated learning systems bring new challenges hindering the development of federated learning techniques. To this end, we propose an Adaptive Asynchronous Federated Learning scheme with Momentum, called FedAAM, comprising an adaptive weight allocation algorithm and a novel asynchronous federated learning framework. Firstly, we dynamically allocate weights for the global model update using an adaptive weight allocation strategy that can improve the convergence rate of models in asynchronous federated learning systems. Then, targeting the challenges mentioned previously, we proposed two new asynchronous global update rules based on the differentiated strategy, which is an essential component of the proposed novel federated learning framework. Furthermore, our asynchronous federated learning framework introduces the historical global update direction (i.e., global momentum) into the global update operation, aiming at improving training efficiency. Moreover, we prove that the model under the FedAAM scheme can achieve a sublinear convergence rate. Extensive experiments on real-world datasets demonstrate that the FedAAM scheme outperforms representative synchronous and asynchronous federated learning schemes (i.e., FedAvg and FedAsync) regarding the model's convergence rate and capacity to deal with dynamic systems.
Original language | English |
---|---|
Pages (from-to) | 193-206 |
Number of pages | 14 |
Journal | Future Generation Computer Systems |
Volume | 152 |
Early online date | 7 Nov 2023 |
DOIs | |
Publication status | Published - Mar 2024 |
Bibliographical note
Funding Information:This work was supported in part by the Joint Funds of the National Natural Science Foundation of China (Grant No. U22A2036 ), the National Key Research and Development Program of China ( 2021YFB3101102 ), the Key-Area Research and Development Program of Guangdong Province, China ( 2020B0101360001 ), and the Fundamental Research Funds for the Central Universities, China ( HIT.OCEF.2021007 ). Professor Weizhe Zhang is the corresponding author.
Publisher Copyright:
© 2023 Elsevier B.V.