A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Advanced Search
Volume 13 Issue 10
Oct.  2026

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 18.3, Top 1 (SCI Q1)
    CiteScore: 28.2, Top 1% (Q1)
    Google Scholar h5-index: 95， TOP 5
Turn off MathJax
Article Contents
Article Navigation >  IEEE/CAA Journal of Automatica Sinica  > 2026  >  13(10): 2300-2311
L. Ji, H. Wang, and C. Zhang, “Optimal consensus of heterogeneous multi-agent system with unknown dynamics,” IEEE/CAA J. Autom. Sinica, vol. 13, no. 10, pp. 2300–2311, Oct. 2026. doi: 10.1109/JAS.2026.125855
Citation: L. Ji, H. Wang, and C. Zhang, “Optimal consensus of heterogeneous multi-agent system with unknown dynamics,” IEEE/CAA J. Autom. Sinica, vol. 13, no. 10, pp. 2300–2311, Oct. 2026. doi: 10.1109/JAS.2026.125855
shu

Optimal Consensus of Heterogeneous Multi-Agent System With Unknown Dynamics

doi: 10.1109/JAS.2026.125855
Funds:  This work was supported in part by the National Natural Science Foundation of China (62276036), the Innovation and Development Joint Fund Project of Chongqing Natural Science Foundation (CSTB2024NSCQ-LZX0118), and the Major Project of Scientific and Technological Research Program of Chongqing Municipal Education Commission (KJZD-M202100602)
More Information
    Abstract
  • This paper addresses the optimal consensus problem for heterogeneous multi-agent systems (MASs) with unknown dynamics. To eliminate dependence on accurate system models, we design distinct performance index functions for first-order and second-order agents, separating their tracking tasks. We employ a reinforcement learning algorithm to estimate the optimal performance index function and control policies. To enhance training stability, a novel training mode and a state-constrained iterative algorithm are proposed, with their convergence and optimality rigorously analyzed. Furthermore, a new error calculation method is introduced to accurately describe agent-neighbor distances, accelerating convergence and reducing energy consumption. The effectiveness of the proposed methods is validated through simulations.

     

    • FullText(HTML)
  • loading
    • References(40)
  • [1]
    Z. Xia, J. Du, J. Wang, C. Jiang, Y. Ren, G. Li, and Z. Han, “Multi-agent reinforcement learning aided intelligent UAV swarm for target tracking,” IEEE Trans. Veh. Technol., vol. 71, no. 1, pp. 931−945, Jan. 2021. doi: 10.1109/iccar57134.2023.10151768
    [2]
    Y. Liu, J. Liu, Z. He, Z. Li, Q. Zhang, and Z. Ding, “A survey of multi-agent systems on distributed formation control,” Unmanned Syst., vol. 12, no. 5, pp. 913−926, Oct. 2024. doi: 10.1142/S2301385024500274
    [3]
    P. R. Mazzarino, A. Macii, L. Bottaccioli, and E. Patti, “A multi-agent framework for smart grid simulations: Strategies for power-to-heat flexibility management in residential context,” Sustainable Energy Grids Networks, vol. 34, Art. no. 101072, 2023. doi: 10.1016/j.segan.2023.101072
    [4]
    C. Hu, G. Wen, S. Wang, J. Fu, and W. Yu, “Distributed multiagent reinforcement learning with action networks for dynamic economic dispatch,” IEEE Trans. Neural Netw. Learn. Syst., vol. 35, no. 7, pp. 9553−9564, Jul. 2024. doi: 10.1109/TNNLS.2023.3234049
    [5]
    Q. Ma, J. Qin, B. D. O. Anderson, and L. Wang, “Exponential consensus of multiple agents over dynamic network topology: Controllability, connectivity, and compactness,” IEEE Trans. Autom. Control, vol. 68, no. 12, pp. 7104−7119, Dec. 2023. doi: 10.1109/TAC.2023.3245021
    [6]
    J. Wang, L. Zhou, D. Zhang, J. Liu, F. Xiao, and Y. Zheng, “Protocol selection for second-order consensus against disturbance,” Automatica, vol. 161, Art. no. 111497, Mar. 2024. doi: 10.1016/j.automatica.2023.111497
    [7]
    D. Wang, Z. Wang, and C. Wen, “Distributed optimal consensus control for a class of uncertain nonlinear multiagent networks with disturbance rejection using adaptive technique,” IEEE Trans. Syst., Man, Cybern. Syst., vol. 51, no. 7, pp. 4389−4399, Jul. 2021. doi: 10.1109/TSMC.2019.2933005
    [8]
    Y. Feng, S. Xu, and B. Zhang, “Group consensus control for double-integrator dynamic multiagent systems with fixed communication topology,” Int. J. Robust Nonlinear Control, vol. 24, no. 3, pp. 532−547, Feb. 2014. doi: 10.1002/rnc.2904
    [9]
    H. Zhang, H. Jiang, Y. Luo, and G. Xiao, “Data-driven optimal consensus control for discrete-time multi-agent systems with unknown dynamics using reinforcement learning method,” IEEE Trans. Ind. Electron., vol. 64, no. 5, pp. 4091−4100, May 2017. doi: 10.1109/TIE.2016.2542134
    [10]
    M. Zhao, C. Peng, W. He, and Y. Song, “Event-triggered communication for leader-following consensus of second-order multiagent systems,” IEEE Trans. Cybern., vol. 48, no. 6, pp. 1888−1897, Jun. 2018. doi: 10.1109/TCYB.2017.2716970
    [11]
    J. Yu and L. Wang, “Group consensus in multi-agent systems with switching topologies and communication delays,” Syst. Control Lett., vol. 59, no. 6, pp. 340−348, Jun. 2010. doi: 10.1016/j.sysconle.2010.03.009
    [12]
    Z. Peng, J. Hu, K. Shi, R. Luo, R. Huang, B. K. Ghosh, and J. Huang, “A novel optimal bipartite consensus control scheme for unknown multi-agent systems via model-free reinforcement learning,” Appl. Math. Comput., vol. 369, Art. no. 124821, Mar. 2020. doi: 10.1016/j.amc.2021.126451
    [13]
    J. Li, L. Ji, C. Zhang, and H. Li, “Optimal couple-group tracking control for the heterogeneous multi-agent systems with cooperative-competitive interactions via reinforcement learning method,” Inf. Sci., vol. 610, pp. 401−424, Sep. 2022. doi: 10.1016/j.ins.2022.07.181
    [14]
    Y. Zheng, Y. Zhu, and L. Wang, “Consensus of heterogeneous multi-agent systems,” IET Control Theory Appl., vol. 5, no. 16, pp. 1881−1888, Nov. 2011. doi: 10.1049/iet-cta.2011.0033
    [15]
    H. Du, G. Wen, D. Wu, Y. Cheng, and J. Lü, “Distributed fixed-time consensus for nonlinear heterogeneous multi-agent systems,” Automatica, vol. 113, Art. no. 108797, Mar. 2020. doi: 10.1016/j.automatica.2019.108797
    [16]
    Y. Zheng and L. Wang, “Finite-time consensus of heterogeneous multi-agent systems with and without velocity measurements,” Syst. Control Lett., vol. 61, no. 8, pp. 871−878, Aug. 2012. doi: 10.1016/j.sysconle.2012.05.009
    [17]
    J. Qin, Q. Ma, H. Gao, Y. Shi, and Y. Kang, “On group synchronization for interacting clusters of heterogeneous systems,” IEEE Trans. Cybern., vol. 47, no. 12, pp. 4122−4133, Dec. 2017. doi: 10.1109/TCYB.2016.2600753
    [18]
    C.-L. Liu and F. Liu, “Dynamical consensus seeking of heterogeneous multi-agent systems under input delays,” Int. J. Commun. Syst., vol. 26, no. 10, pp. 1243−1258, Oct. 2013. doi: 10.1002/dac.1393
    [19]
    G. Wen, Y. Yu, Z. Peng, and H. Wang, “Dynamical group consensus of heterogenous multi-agent systems with input time delays,” Neurocomputing, vol. 175, pp. 278−286, Jan. 2016. doi: 10.1016/j.neucom.2015.10.060
    [20]
    F. Yu, L. Ji, and S. Yang, “Group consensus for a class of heterogeneous multi-agent networks in the competition systems,” Neurocomputing, vol. 416, pp. 165−171, Nov. 2020. doi: 10.1016/j.neucom.2019.05.091
    [21]
    W. Li, X. Du, J. Xiao, and L. Zhang, “Bipartite hybrid formation tracking control for heterogeneous multi-agent systems in multi-group cooperative-competitive networks,” Appl. Math. Comput., vol. 456, Art. no. 128133, Nov. 2023. doi: 10.1016/j.amc.2023.128133
    [22]
    F. Sun, Y. Han, W. Zhu, and J. Kurths, “Group consensus for fractional-order heterogeneous multi-agent systems under cooperation-competition networks with time delays,” Commun. Nonlinear Sci. Numer. Simul., vol. 133, Art. no. 107951, Jun. 2024. doi: 10.1016/j.cnsns.2024.107951
    [23]
    J. Zhang, Y. Fu, and J. Fu, “Optimal formation control of second-order heterogeneous multiagent systems using adaptive predefined-time strategy,” IEEE Trans. Fuzzy Syst., vol. 32, no. 4, pp. 2390−2402, Apr. 2024. doi: 10.1109/TFUZZ.2024.3351993
    [24]
    L. Ji, C. Wang, C. Zhang, H. Wang, and H. Li, “Optimal consensus model-free control for multi-agent systems subject to input delays and switching topologies,” Inf. Sci., vol. 589, pp. 497−515, Apr. 2022. doi: 10.1016/j.ins.2021.12.125
    [25]
    C. Zhang, L. Ji, S. Yang, X. Guo, J. Li, and H. Li, “Time-varying formation optimization tracking of multi-agent systems with semi-Markov switching topology,” Nonlinear Dyn., vol. 112, no. 12, pp. 10095−10108, Apr. 2024. doi: 10.1007/s11071-024-09599-4
    [26]
    H. Li and Q. Wei, “Data-driven optimal output cluster synchronization control of heterogeneous multi-agent systems,” IEEE Trans. Autom. Sci. Eng., vol. 21, no. 3, pp. 3910−3920, Jul. 2024. doi: 10.1109/TASE.2023.3289950
    [27]
    A. Heydari, “Stability analysis of optimal adaptive control under value iteration using a stabilizing initial policy,” IEEE Trans. Neural Netw. Learn. Syst., vol. 29, no. 9, pp. 4522−4527, Sep. 2018. doi: 10.1109/TNNLS.2017.2755501
    [28]
    B. Luo, Y. Yang, H.-N. Wu, and T. Huang, “Balancing value iteration and policy iteration for discrete-time control,” IEEE Trans. Syst., Man, Cybern. Syst., vol. 50, no. 11, pp. 3948−3958, Nov. 2020. doi: 10.1109/TSMC.2019.2898389
    [29]
    D. Liu and Q. Wei, “Policy iteration adaptive dynamic programming algorithm for discrete-time nonlinear systems,” IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 3, pp. 621−634, Mar. 2014. doi: 10.1109/TNNLS.2013.2281663
    [30]
    M. Fang, Y. Li, and T. Cohn, “Learning how to active learn: A deep reinforcement learning approach,” in Proc. Conf. Empirical Methods in Natural Language Processing, Copenhagen, Denmark, 2017, pp. 595−605.
    [31]
    B. Luo, Y. Yang, and D. Liu, “Adaptive Q-learning for data-based optimal output regulation with experience replay,” IEEE Trans. Cybern., vol. 48, no. 12, pp. 3337−3348, Dec. 2018. doi: 10.1109/TCYB.2018.2821369
    [32]
    C. Sun, X. Li, and Y. Sun, “A parallel framework of adaptive dynamic programming algorithm with off-policy learning,” IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 8, pp. 3578−3587, Aug. 2021. doi: 10.1109/TNNLS.2020.3015767
    [33]
    M. Lin, B. Zhao, and D. Liu, “Policy gradient adaptive critic designs for model-free optimal tracking control with experience replay,” IEEE Trans. Syst., Man, Cybern. Syst., vol. 52, no. 6, pp. 3692−3703, Jun. 2022. doi: 10.1109/TSMC.2021.3071968
    [34]
    L. Ji, Z. Lin, C. Zhang, S. Yang, J. Li, and H. Li, “Data-based optimal consensus control for multiagent systems with time delays: Using prioritized experience replay,” IEEE Trans. Syst., Man, Cybern. Syst., vol. 54, no. 5, pp. 3244−3256, May 2024. doi: 10.1109/TSMC.2024.3358293
    [35]
    J. Peters, “Policy gradient methods,” Scholarpedia, vol. 5, no. 11, Art. no. 3698, 2010. doi: 10.1007/springerreference_179352
    [36]
    D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, and M. Riedmiller, “Deterministic policy gradient algorithms,” in Proc. 31st Int. Conf. Machine Learning, Beijing, China, 2014, pp. 387−395.
    [37]
    T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” in Proc. 4th Int. Conf. Learning Representations. San Juan, Puerto Rico, 2016.
    [38]
    H. Shen, C. Peng, H. Yan, and S. Xu, “Data-driven near optimization for fast sampling singularly perturbed systems,” IEEE Trans. Autom. Control, vol. 69, no. 7, pp. 4689−4694, Jul. 2024. doi: 10.1109/TAC.2024.3352703
    [39]
    H. Shen, Y. Wang, H. Yan, and S. Xu, “Data-driven single-loop policy iteration control of uncertain singularly perturbed systems,” IEEE Trans. Autom. Control, vol. 70, no. 12, pp. 8314−8320, Dec. 2025. doi: 10.1109/TAC.2025.3581141
    [40]
    K. Zhang, X. Dong, H. Zhang, and R. Su, “Zero-sum optimal control for a cyber-physical system in unreliable communications via secure decentralized policy iteration,” IEEE Internet Things J., vol. 12, no. 14, pp. 27339−27349, Jul. 2025. doi: 10.1109/JIOT.2025.3562385
    • Related Articles
    • Supplements(0)

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(12)

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (14) PDF downloads(0) Cited by()

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return