A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Volume 7 Issue 6
Oct.  2020

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 11.8, Top 4% (SCI Q1)
    CiteScore: 17.6, Top 3% (Q1)
    Google Scholar h5-index: 77, TOP 5
Turn off MathJax
Article Contents
Jingwei Lu, Qinglai Wei and Fei-Yue Wang, "Parallel Control for Optimal Tracking via Adaptive Dynamic Programming," IEEE/CAA J. Autom. Sinica, vol. 7, no. 6, pp. 1662-1674, Nov. 2020. doi: 10.1109/JAS.2020.1003426
Citation: Jingwei Lu, Qinglai Wei and Fei-Yue Wang, "Parallel Control for Optimal Tracking via Adaptive Dynamic Programming," IEEE/CAA J. Autom. Sinica, vol. 7, no. 6, pp. 1662-1674, Nov. 2020. doi: 10.1109/JAS.2020.1003426

Parallel Control for Optimal Tracking via Adaptive Dynamic Programming

doi: 10.1109/JAS.2020.1003426
Funds:

the National Key Reseanch and Development Program of China 2018AAA0101502

the National Key Reseanch and Development Program of China 2018YFB1702300

the National Natural Science Foundation of China 61722312

the National Natural Science Foundation of China 61533019

the National Natural Science Foundation of China U1811463

the National Natural Science Foundation of China 61533017

More Information
  • This paper studies the problem of optimal parallel tracking control for continuous-time general nonlinear systems. Unlike existing optimal state feedback control, the control input of the optimal parallel control is introduced into the feedback system. However, due to the introduction of control input into the feedback system, the optimal state feedback control methods can not be applied directly. To address this problem, an augmented system and an augmented performance index function are proposed firstly. Thus, the general nonlinear system is transformed into an affine nonlinear system. The difference between the optimal parallel control and the optimal state feedback control is analyzed theoretically. It is proven that the optimal parallel control with the augmented performance index function can be seen as the suboptimal state feedback control with the traditional performance index function. Moreover, an adaptive dynamic programming (ADP) technique is utilized to implement the optimal parallel tracking control using a critic neural network (NN) to approximate the value function online. The stability analysis of the closed-loop system is performed using the Lyapunov theory, and the tracking error and NN weights errors are uniformly ultimately bounded (UUB). Also, the optimal parallel controller guarantees the continuity of the control input under the circumstance that there are finite jump discontinuities in the reference signals. Finally, the effectiveness of the developed optimal parallel control method is verified in two cases.

     

  • loading
  • Recommended by Associate Editor Mangchu Zhou.
  • [1]
    F.-Y. Wang, "Parallel system methods for management and control of complex systems, " Control and Decision, vol. 19, no. 5, pp. 485-489, May 2004. https://www.researchgate.net/publication/266537911_Parallel_system_methods_for_management_and_control_of_complex_systems
    [2]
    F.-Y. Wang, "Parallel control and management for intelligent transportation systems: concepts, architectures, and applications, " IEEE Trans. Intell. Transp. Syst., vol. 11, no. 3, pp. 630-638, Sep. 2010. https://www.researchgate.net/publication/220108880_Parallel_Control_and_Management_for_Intelligent_Transportation_Systems_Concepts_Architectures_and_Applications
    [3]
    Q. Wei, D. Liu, and H. Lin, "Value iteration adaptive dynamic programming for optimal control of discrete-time nonlinear systems, " IEEE Trans. Cybern., vol. 46, no. 3, pp. 840-853, Mar. 2016. https://www.researchgate.net/publication/283624374_Value_Iteration_Adaptive_Dynamic_Programming_for_Optimal_Control_of_Discrete-Time_Nonlinear_Systems
    [4]
    D. Liu, Q. Wei, D. Wang, X. Yang, and H. Li, Adaptive Dynamic Programming with Applications in Optimal Control. Cham, Switzerland: Springer, 2017.
    [5]
    Y. Li, T, Yang, and S. Tong, "Adaptive neural networks finite-time optimal control for a class of nonlinear systems, " IEEE Trans. Neural. Netw. Learn. Syst., in press, doi: 10.1109/TNNLS.2019.2955438
    [6]
    H. Zhang, L. Jiang, J. Liu, and F. Qu, "Data recovery of magnetic flux leakage data gaps using multifeature conditional risk, " IEEE Trans. Autom. Sci. Eng.,, in press, doi: 10.1109/TASE.2020.2994659
    [7]
    Q. Wei, R. Song, Z. Liao, B. Li, and F. L. Lewis, "Discrete-time impulsive adaptive dynamic programming, " IEEE Trans. Cybern., in press, doi: 10.1109/TCYB.2019.2906694
    [8]
    R. Song, J. Li, F. L. Lewis, "Robust optimal control for disturbed nonlinear zero-sum differential games based on Single NN and Least-Squares, " IEEE Trans. Syst., Man, Cybern., Syst., in press, doi: 10.1109/TSMC.2019.2897379
    [9]
    R. Song, Q. L. Wei, H. Zhang, F. L. Lewis, "Discrete-time non-zero-sum games with completely unknown dynamics, " IEEE Trans. Cybern., in press, doi: 10.1109/TCYB.2019.2957406
    [10]
    P. J. Werbos, "Advanced forecasting methods for global crisis warning and models of intelligence, " Gen. Syst. Yearbook, vol. 22, pp. 25-38, 1977.
    [11]
    P. J. Werbos, "Approximate dynamic programming for real-time control and neural modeling, " in Handbook of Intelligent Control, D. A. White and D. A. Sofge, Eds. New York: Van Nostrand Reinhold, 1992. https://www.researchgate.net/publication/243651581_Approximate_dynamic_programming_for_real-time_control_and_neural_modeling
    [12]
    F. L. Lewis, D. Vrabie, and V. L. Syrmos, Optimal Control. Hoboken, NJ, USA: Wiley, 2013.
    [13]
    H. Zhang, K. Zhang, G. Xiao, and H. Jiang, "Robust optimal control scheme for unknown constrained-input nonlinear systems via a plug-n-play event-sampled critic-only algorithm, " IEEE Trans. Syst., Man, Cybern., Syst.,, in press, doi: 10.1109/TSMC.2018.2889377
    [14]
    Q. Wei, Z. Liao, R. Song, P. Zhang, Z. Wang, and J. Xiao, "Self-learning optimal control for ice storage air conditioning systems via data-based adaptive dynamic programming, " IEEE Trans. Ind. Electron., in press 2020. DOI: 10.1109/TIE.2020.2978699
    [15]
    Y.-M. Li, X. Min, and S. Tong, "Adaptive fuzzy inverse optimal control for uncertain strict-feedback nonlinear systems, " IEEE Trans. Neural. Netw. Learn. Syst., vol. 28, no. 10, pp. 2363-2374, Oct. 2020.
    [16]
    A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, "Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof, " IEEE Trans. Syst., Man, Cybern. B. Cybern., vol. 38, no. 4, pp. 943-949, Aug. 2008. http://med.wanfangdata.com.cn/Paper/Detail/PeriodicalPaper_PM18632382
    [17]
    F. L. Lewis and D. Vrabie, "Reinforcement learning and adaptive dynamic programming for feedback control, " IEEE Circuits Syst. Mag., vol. 9, no. 3, pp. 32-50, Jul. 2009. Reinforcement learning and adaptive dynamic programming for feedback control
    [18]
    F.-Y. Wang, N. Jin, D. Liu, and Q. Wei, "Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with ε-error bound, " IEEE Trans. Neural. Netw. Learn. Syst., vol. 22, no. 1, pp. 24-36, Jan. 2011. http://med.wanfangdata.com.cn/Paper/Detail/PeriodicalPaper_PM20876014
    [19]
    T. Dierks and S. Jagannathan, "Online optimal control of affine nonlinear discrete-time systems with unknown internal dynamics by using time-based policy update, " IEEE Trans. Neural. Netw. Learn. Syst., vol. 23, no. 7, pp. 1118-1129, Jul. 2012. https://pubmed.ncbi.nlm.nih.gov/24807137/
    [20]
    C. Mu, D. Wang, and H. He, "Data-driven finite-horizon approximate optimal control for discrete-time nonlinear systems using iterative HDP approach, " IEEE Trans. Cybern., vol. 48, no. 10, pp. 2948-2961, Oct. 2018. https://pubmed.ncbi.nlm.nih.gov/29028219/
    [21]
    L. Dong, J. Yan, X. Yuan, H. He, and C. Sun, "Functional nonlinear model predictive control based on adaptive dynamic programming, " IEEE Trans. Cybern., vol. 49, no. 12, pp. 4206-4218, Dec. 2019. https://www.researchgate.net/publication/327089881_Functional_Nonlinear_Model_Predictive_Control_Based_on_Adaptive_Dynamic_Programming
    [22]
    Q. Wei, Z. Liao, Z. Yang, B. Li, and D. Liu, "Continuous-time time-varying policy iteration, " IEEE Trans. Cybern., in press, doi: 10.1109/TCYB.2019.2926631
    [23]
    Q. Wei, H. Li, X. Yang, and H. He, "Continuous-time distributed policy iteration for multicontroller nonlinear systems, " IEEE Trans. Cybern., in press, doi: 10.1109/TCYB.2020.2979614
    [24]
    Q. Wei, H. Li, X. Yang, and H. He, "Continuous-time distributed policy iteration for multicontroller nonlinear systems, " IEEE Trans. Cybern., in press, doi: 10.1109/TCYB.2020.2979614
    [25]
    H. Zhang, Y. Cai, Y. Wang, and H. Su, "Adaptive bipartite event-triggered output consensus of heterogeneous linear multiagent systems under fixed and switching topologies, " IEEE Trans. Neural Netw. Learn. Syst., in press, doi: 10.1109/TNNLS.2019.2958107
    [26]
    Q. Wei, L. Wang, Y. Liu, and M. M. Polycarpou, "Optimal elevator group control via deep asynchronous actor-critic learning, " IEEE Trans. Neural. Netw. Learn. Syst., in press, doi: 10.1109/TNNLS.2020.2965208
    [27]
    Q. Wei, D. Liu, Y. Liu, and R. Song, "Optimal Constrained Self-learning Battery Sequential Management in Microgrid Via Adaptive Dynamic Programming, " IEEE/CAA J. Autom. Sinica, vol. 4, no. 2, pp. 168-176, Apr. 2017. http://en.cnki.com.cn/Article_en/CJFDTOTAL-ZDHB201702003.htm
    [28]
    D. Liu, Y. Xu, Q. Wei, and X. Liu, "Residential energy scheduling for variable weather solar energy based on adaptive dynamic programming, " IEEE/CAA J. Autom. Sinica, vol. 5, no. 1, pp. 36-46, Jan. 2018. http://en.cnki.com.cn/Article_en/CJFDTotal-ZDHB201801004.htm
    [29]
    K. G. Vamvoudakis and F. L. Lewis, "Online actorritic algorithm to solve the continuous-time infinite horizon optimal control problem, " Automatica, vol. 46, no. 5, pp. 878-808, 2010. doi: 10.1016/j.automatica.2010.02.018
    [30]
    T. Dierks and S. Jagannathan, "Optimal control of affine nonlinear continuous-time systems, " in Proc. Amer. Control Conf., MD, USA, Jul. 2010, pp. 1568-1573. https://www.researchgate.net/publication/224162610_Optimal_control_of_affine_nonlinear_continuous-time_systems
    [31]
    D. Nodland, H. Zargarzadeh, and S. Jagannathan, "Neural network-based optimal adaptive output feedback control of a helicopter UAV, " IEEE Trans. Neural. Netw. Learn. Syst., vol. 24, no. 7, pp. 1061-1073, Jul. 2013. https://pubmed.ncbi.nlm.nih.gov/24808521/
    [32]
    H. Zargarzadeh, T. Dierks, and S. Jagannathan, "Optimal control of nonlinear continuous-time systems in strict-feedback form, " IEEE Trans. Neural. Netw. Learn. Syst., vol. 26, no. 10, pp. 2535-2549, Oct. 2015. https://pubmed.ncbi.nlm.nih.gov/26111400/
    [33]
    R. Bell and K. Aström, Dynamic Models for Boliler-Turbine-Alternator Units: Data Logs and Parameter Estimation for a 160 MW Unit. Sweden: Department of Automatic Control, Lund Institute of Technology, 1987.
    [34]
    G. Pellegrinetti, and J. Bentsman, "Nonlinear control oriented boiler modeling-a benchmark problem for controller design, " IEEE Trans. Control Syst. Technol., vol. 4, no. 1, pp. 57-64, Jan. 1996. https://www.researchgate.net/publication/3331856_Nonlinear_control_oriented_boiler_modeling_-_A_benchmark_problem_for_controller_design
    [35]
    G. Wen, S. S. Ge, W. Zhou, C. L. P. Chen, F. Tu, and S. Wang, "Adaptive tracking control of surface vessel using optimized backstepping technique, " IEEE Trans. Cybern., vol. 49, no. 9, pp. 3420-3431, Sep. 2019. https://ieeexplore.ieee.org/document/8396836
    [36]
    W. Gao, and Z. P. Jiang, "Learning-based adaptive optimal tracking control of strict-feedback nonlinear systems, " IEEE Trans. Neural. Netw. Learn. Syst., vol. 29, no. 6, pp. 2614-2624, Jun. 2018. https://pubmed.ncbi.nlm.nih.gov/29771677/
    [37]
    Y. Liu, H. Zhang, R. Yu, and Z. Xing, "H Tracking control of discrete-time system with delays via data-based adaptive dynamic programming, " IEEE Trans. Syst., Man, Cybern., Syst., to be published, doi: 10.1109/TSMC.2019.2946397.
    [38]
    F. Li, M. Zhou, and Y. Ding, "An adaptive online co-search method with distributed samples for dynamic target tracking, " IEEE Trans. Control Syst. Technol., vol. 26, no. 2, pp. 439-451, Mar. 2018.
    [39]
    S. Li, M. Zhou, and X. Luo, "Modified primal-dual neural networks for motion control of redundant manipulators with dynamic rejection of harmonic noises, " IEEE Trans. Neural. Netw. Learn. Syst., vol. 29, no. 10, pp. 4791-4801, Oct. 2018. https://www.researchgate.net/publication/322093421_Modified_Primal-Dual_Neural_Networks_for_Motion_Control_of_Redundant_Manipulators_With_Dynamic_Rejection_of_Harmonic_Noises
    [40]
    H. Zhang, Q. Wei, and Y. Luo, "A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm, " IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 937-942, Aug. 2008.
    [41]
    H. Zhang, L. Cui, X. Zhang, and Y. Luo, "Data-driven robust approximate optimal tracking control for unknown general nonlinear systems using adaptive dynamic programming method, " IEEE Trans. Neural. Netw. Learn. Syst., vol. 22, no. 12, pp. 2226-2236, Dec. 2011. http://med.wanfangdata.com.cn/Paper/Detail/PeriodicalPaper_PM21997259
    [42]
    C. Mu, Z. Ni, C. Sun, and H. He, "Data-driven tracking control with adaptive dynamic programming for a class of continuous-Time Nonlinear Systems, " IEEE Trans. Cybern., vol. 47, no. 6, pp. 1460-1470, Jun. 2017. https://pubmed.ncbi.nlm.nih.gov/27116758/
    [43]
    Z. Cao, Q. Xiao, R. Huang, and M. Zhou, "Robust neuro-optimal control of underactuated snake robots with experience replay, " IEEE Trans. Neural. Netw. Learn. Syst., vol. 29, no. 1, pp. 208-217, Jan. 2018. https://www.researchgate.net/publication/321091006_Robust_Neuro-Optimal_Control_of_Underactuated_Snake_Robots_With_Experience_Replay
    [44]
    Q. Wei, H. Li, and F.-Y. Wang, "Parallel control for continuous-time linear systems: A case study, " IEEE/CAA J. Autom. Sinica, vol. 7, no. 4, pp. 919-926, Jul. 2020.
    [45]
    F.-Y. Wang, "Parallel control: a method for data-driven and computation control, " Acta Automatica Sinica, vol. 39, no. 4, pp. 293-302, Apr. 2013. http://en.cnki.com.cn/Article_en/CJFDTOTAL-MOTO201304002.htm
    [46]
    F.-Y. Wang and Q. Wei, "Intelligent control: from learning control to parallel control, " Control Theory & Applications, vol. 35, no. 7, pp. 939-948, Jul. 2018. http://en.cnki.com.cn/Article_en/CJFDTotal-KZLY201807005.htm
    [47]
    F. Zhu, Z. Li, S. Chen, and G. Xiong, "Parallel transportation management and control system and its applications in building smart cities, " IEEE Trans. Intell. Transp. Syst., vol. 17, no. 6, pp. 1576-1585, Jun. 2016. https://ieeexplore.ieee.org/document/7381677
    [48]
    B. Ning, H. Dong, D. Weng, L. Li, and C.-J. Cheng, "ACP-based control and management of urban rail transportation systems, " IEEE Intell. Syst., vol. 26, no. 2, pp. 84-88, Mar. 2011. https://ieeexplore.ieee.org/document/5751217
    [49]
    F.-Y. Wang and S. Tang, "Artificial societies for integrated and sustainable development of metropolitan systems, " IEEE Intell. Syst., vol. 19, no. 4, pp. 82-87, Jul. 2004. https://www.researchgate.net/publication/220628871_Artificial_Societies_for_Integrated_and_Sustainable_Development_of_Metropolitan_Systems
    [50]
    F.-Y. Wang, N.-N Zheng, D. Cao, C. M. Martinez, L. Li, and T. Liu, "Parallel driving in CPSS: A unified approach for transport automation and vehicle intelligence, " IEEE/CAA J. Autom. Sinica, vol. 4, no. 4, pp. 577-587, Oct. 2017. https://ieeexplore.ieee.org/document/8039015
    [51]
    W. Zhang, K. Wang, Y. Liu, Y. Lu, and F.-Y. Wang, "A parallel vision approach to scene-specific pedestrian detection, " Neurocomputing, vol. 394, no. 21, pp. 114-126, Jun. 2020. https://www.researchgate.net/publication/334007978_A_Parallel_Vision_Approach_to_Scene-Specific_Pedestrian_Detection
    [52]
    Q. Wei, L. Wang, J. Lu, and F.-Y. Wang, "Discrete-Time self-learning parallel control, " IEEE Trans. Syst., Man, Cybern., Syst., in press, doi: 10.1109/TSMC.2020.2995646.
    [53]
    G. Bartolini and P. Pydynowski, "An improved, chattering free, V.S.C. scheme for uncertain dynamical systems, " IEEE Trans. Autom. Control, vol. 41, no. 8, pp. 1220-1226, Aug. 1996. https://ieeexplore.ieee.org/document/533691
    [54]
    M. Krstic, I. Kanellakopoulos, and P. V. Kokotovic, Nonlinear and Adaptive Control Design. New York, USA: John Wiley & Sons. Inc., 1995.
    [55]
    F. L. Lewis, S. Jagannathan, and A. Yesildirek, Neural Network Control of Robot Manipulators and Nonlinear Systems. Philadelphia, USA: Taylor & Francis, 1999.
    [56]
    P. Tabuada, "Event-triggered real-time scheduling of stabilizing control tasks, " IEEE Trans. Autom. Control, vol. 52, no. 9, pp. 1680-1685, Sep. 2007. https://ieeexplore.ieee.org/document/4303247

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(11)

    Article Metrics

    Article views (1290) PDF downloads(150) Cited by()

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return