A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 15.3, Top 1 (SCI Q1)
    CiteScore: 23.5, Top 2% (Q1)
    Google Scholar h5-index: 77, TOP 5
Turn off MathJax
Article Contents
B. Esmaeili and H. Modares, “Risk-informed model-free safe control of linear parameter-varying systems,” IEEE/CAA J. Autom. Sinica, vol. 11, no. 9, pp. 1–15, Sept. 2024. doi: 10.1109/JAS.2024.124479
Citation: B. Esmaeili and H. Modares, “Risk-informed model-free safe control of linear parameter-varying systems,” IEEE/CAA J. Autom. Sinica, vol. 11, no. 9, pp. 1–15, Sept. 2024. doi: 10.1109/JAS.2024.124479

Risk-Informed Model-Free Safe Control of Linear Parameter-Varying Systems

doi: 10.1109/JAS.2024.124479
Funds:  This work was supported in part by the Department of Navy award (N00014-22-1-2159) and the National Science Foundation under award (ECCS-2227311)
More Information
  • This paper presents a risk-informed data-driven safe control design approach for a class of stochastic uncertain nonlinear discrete-time systems. The nonlinear system is modeled using linear parameter-varying (LPV) systems. A model-based probabilistic safe controller is first designed to guarantee probabilistic $\lambda$-contractivity (i.e., stability and invariance) of the LPV system with respect to a given polyhedral safe set. To obviate the requirement of knowing the LPV system model and to bypass identifying its open-loop model, its closed-loop data-based representation is provided in terms of state and scheduling data as well as a decision variable. It is shown that the variance of the closed-loop system, as well as the probability of safety satisfaction, depends on the decision variable and the noise covariance. A minimum-variance direct data-driven gain-scheduling safe control design approach is presented next by designing the decision variable such that all possible closed-loop system realizations satisfy safety with the highest confidence level. This minimum-variance approach is a control-oriented learning method since it minimizes the variance of the state of the closed-loop system with respect to the safe set, and thus minimizes the risk of safety violation. Unlike the certainty-equivalent approach that results in a risk-neutral control design, the minimum-variance method leads to a risk-averse control design. It is shown that the presented direct risk-averse learning approach requires weaker data richness conditions than existing indirect learning methods based on system identification and can lead to a lower risk of safety violation. Two simulation examples along with an experimental validation on an autonomous vehicle are provided to show the effectiveness of the presented approach.

     

  • loading
  • 1 To watch the animation of the path tracking performance, please click on the following link: Safe path tracking animation.
    2 To watch the video of the set-point tracking performance of the robot in Gazebo, please click on the following link: Safe set-point tracking simulation.
    3 To watch the video of the set-point tracking performance of the robot in real-world, please click on the following link: Safe set-point tracking implementation.
  • [1]
    L. Brunke, M. Greeff, A. W. Hall, Z. Yuan, S. Zhou, J. Panerati, and A. P. Schoellig, “Safe learning in robotics: From learning-based control to safe reinforcement learning,” Annu. Rev. Control Robot. Auton. Syst., vol. 5, pp. 411–444, May 2022. doi: 10.1146/annurev-control-042920-020211
    [2]
    M. Zanon and S. Gros, “Safe reinforcement learning using robust MPC,” IEEE Trans. Autom. Control, vol. 66, no. 8, pp. 3638–3652, Aug. 2021. doi: 10.1109/TAC.2020.3024161
    [3]
    R. Grandia, A. J. Taylor, A. D. Ames, and M. Hutter, “Multi-layered safety for legged robots via control barrier functions and model predictive control,” in Proc. IEEE Int. Conf. Robotics and Automation, Xi'an, China, 2021, pp. 8352-8358.
    [4]
    M. Mazouchi, S. Nageshrao, and H. Modares, “Conflict-aware safe reinforcement learning: A meta-cognitive learning framework,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 3, pp. 466–481, Mar. 2022. doi: 10.1109/JAS.2021.1004353
    [5]
    L. Zhang, R. Zhang, T. Wu, R. Weng, M. Han, and Y. Zhao, “Safe reinforcement learning with stability guarantee for motion planning of autonomous vehicles,” IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 12, pp. 5435–5444, Dec. 2021. doi: 10.1109/TNNLS.2021.3084685
    [6]
    M. Alshiekh, R. Bloem, R. Ehlers, B. Könighofer, S. Niekum, and U. Topcu, “Safe reinforcement learning via shielding,” in Proc. 32nd AAAI Conf. Artificial Intelligence, New Orleans, USA, 2018, pp. 2669–2678.
    [7]
    S. Li and O. Bastani, “Robust model predictive shielding for safe reinforcement learning with stochastic dynamics,” in Proc. IEEE Int. Conf. Robotics and Automation, Paris, France, 2020, pp. 7166–7172.
    [8]
    S. Gao, Z. Peng, H. Wang, L. Liu, and D. Wang, “Safety-critical model-free control for multi-target tracking of USVs with collision avoidance,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 7, pp. 1323–1326, Jul. 2022. doi: 10.1109/JAS.2022.105707
    [9]
    G. Yang, C. Belta, and R. Tron, “Self-triggered control for safety critical systems using control barrier functions,” in Proc. American Control Conf., Philadelphia, USA, 2019, pp. 4454–4459.
    [10]
    Z. Marvi and B. Kiumarsi, “Barrier-certified learning-enabled safe control design for systems operating in uncertain environments,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 3, pp. 437–449, Mar. 2021.
    [11]
    M. Ahmadi, X. Xiong, and A. D. Ames, “Risk-averse control via CVaR barrier functions: Application to bipedal robot locomotion,” IEEE Control Syst. Lett., vol. 6, pp. 878–883, 2022. doi: 10.1109/LCSYS.2021.3086854
    [12]
    R. Cheng, G. Orosz, R. M. Murray, and J. W. Burdick, “End-to-end safe reinforcement learning through barrier functions for safety-critical continuous control tasks,” in Proc. 33rd AAAI Conf. Artificial Intelligence, Honolulu, USA, 2019, pp. 3387–3395.
    [13]
    J. Zeng, B. Zhang, and K. Sreenath, “Safety-critical model predictive control with discrete-time control barrier function,” in Proc. American Control Conf., New Orleans, USA, 2021, pp. 3882–3889.
    [14]
    J. Seo, J. Lee, E. Baek, R. Horowitz, and J. Choi, “Safety-critical control with nonaffine control inputs via a relaxed control barrier function for an autonomous vehicle,” IEEE Robot. Autom. Lett., vol. 7, no. 2, pp. 1944–1951, Apr. 2022. doi: 10.1109/LRA.2022.3142408
    [15]
    A. Chern, X. Wang, A. Iyer, and Y. Nakahira, “Safe control in the presence of stochastic uncertainties,” in Proc. 60th IEEE Conf. Decision and Control, Austin, USA, 2021, pp. 6640–6645.
    [16]
    A. Agrawal and K. Sreenath, “Discrete control barrier functions for safety-critical control of discrete systems with application to bipedal robot navigation,” in Proc. Robotics: Science and Systems, Cambridge, USA, 2017.
    [17]
    S. Samuelson and I. Yang, “Safety-aware optimal control of stochastic systems using conditional value-at-risk,” in Proc. Annu. American Control Conf., Milwaukee, USA, 2018, pp. 6285–6290.
    [18]
    M. F. Reis, A. P. Aguiar, and P. Tabuada, “Control barrier function-based quadratic programs introduce undesirable asymptotically stable equilibria,” IEEE Control Syst. Lett., vol. 5, no. 2, pp. 731–736, Apr. 2021. doi: 10.1109/LCSYS.2020.3004797
    [19]
    Y. Lan and I. Mezić, “Linearization in the large of nonlinear systems and Koopman operator spectrum,” Phys. D Nonlinear Phenom., vol. 242, no. 1, pp. 42–53, Jan. 2013. doi: 10.1016/j.physd.2012.08.017
    [20]
    B. Esmaeili, M. Salim, and M. Baradarannia, “Predefined performance-based model-free adaptive fractional-order fast terminal sliding-mode control of MIMO nonlinear systems,” ISA Trans., vol. 131, pp. 108–123, Dec. 2022. doi: 10.1016/j.isatra.2022.05.036
    [21]
    B. Esmaeili, S. S. Madani, M. Salim, M. Baradarannia, and S. Khanmohammadi, “Model-free adaptive iterative learning integral terminal sliding mode control of exoskeleton robots,” J. Vib. Control, vol. 28, no. 21-22, pp. 3120–3139, Nov. 2022. doi: 10.1177/10775463211026031
    [22]
    F. Blanchini and S. Miani, Set-Theoretic Methods in Control. Boston, USA: Springer, 2008.
    [23]
    Z. Gao and J. Fu, “Robust LPV modeling and control of aircraft flying through wind disturbance,” Chin. J. Aeronaut., vol. 32, no. 7, pp. 1588–1602, Jul. 2019. doi: 10.1016/j.cja.2019.03.029
    [24]
    K. Zhu, D. Ma, and J. Zhao, “Event triggered control for a switched LPV system with applications to aircraft engines,” IET Control Theory Appl., vol. 12, no. 10, pp. 1505–1514, Jul. 2018. doi: 10.1049/iet-cta.2017.0895
    [25]
    A. San-Miguel, V. Puig, and G. Alenyà, “Disturbance observer-based LPV feedback control of a N-DoF robotic manipulator including compliance through gain shifting,” Control Eng. Pract., vol. 115, p. 104887, Oct. 2021. doi: 10.1016/j.conengprac.2021.104887
    [26]
    P. S. G. Cisneros, A. Sridharan, and H. Werner, “Constrained predictive control of a robotic manipulator using quasi-LPV representations,” IFAC-PapersOnLine, vol. 51, no. 26, pp. 118–123, 2018. doi: 10.1016/j.ifacol.2018.11.158
    [27]
    M. A. H. Darwish, P. B. Cox, I. Proimadis, G. Pillonetto, and R. Tóth, “Prediction-error identification of LPV systems: A nonparametric Gaussian regression approach,” Automatica, vol. 97, pp. 92–103, Nov. 2018. doi: 10.1016/j.automatica.2018.07.032
    [28]
    A. Bisoffi, C. De Persis, and P. Tesi, “Data-based guarantees of set invariance properties,” IFAC-PapersOnLine, vol. 53, no. 2, pp. 3953–3958, 2020. doi: 10.1016/j.ifacol.2020.12.2250
    [29]
    A. Luppi, C. De Persis, and P. Tesi, “On data-driven stabilization of systems with nonlinearities satisfying quadratic constraints,” Syst. Control Lett., vol. 163, p. 105206, May 2022. doi: 10.1016/j.sysconle.2022.105206
    [30]
    A. Bisoffi, C. De Persis, and P. Tesi, “Controller design for robust invariance from noisy data,” IEEE Trans. Autom. Control, vol. 68, no. 1, pp. 636–643, Jan. 2023. doi: 10.1109/TAC.2022.3170373
    [31]
    C. De Persis and P. Tesi, “Low-complexity learning of linear quadratic regulators from noisy data,” Automatica, vol. 128, p. 109548, Jun. 2021. doi: 10.1016/j.automatica.2021.109548
    [32]
    H. Modares, “Data-driven safe control of uncertain linear systems under aleatory uncertainty,” IEEE Trans. Autom. Control, vol. 69, no. 1, pp. 551–558, Jan. 2024. doi: 10.1109/TAC.2023.3267019
    [33]
    J. M. Snider, “Automatic steering methods for autonomous automobile path tracking,” Carnegie Mellon University, Pittsburgh, USA, Tech. Rep. CMU-RI-TR-09-08, 2009.
    [34]
    E. Alcala, V. Puig, J. Quevedo, and T. Escobet, “Gain-scheduling LPV control for autonomous vehicles including friction force estimation and compensation mechanism,” IET Control Theory Appl., vol. 12, no. 12, pp. 1683–1693, Aug. 2018. doi: 10.1049/iet-cta.2017.1154
    [35]
    E. Alcalá, V. Puig, and J. Quevedo, “LPV-MPC control for autonomous vehicles,” IFAC-PapersOnLine, vol. 52, no. 28, pp. 106–113, 2019. doi: 10.1016/j.ifacol.2019.12.356
    [36]
    X. Geng and L. Xie, “Data-driven decision making in power systems with probabilistic guarantees: Theory and applications of chance-constrained optimization,” Annu. Rev. Control, vol. 47, pp. 341–363, 2019. doi: 10.1016/j.arcontrol.2019.05.005
    [37]
    A. Marcos and G. J. Balas, “Development of linear-parameter-varying models for aircraft,” J. Guid. Control Dyn., vol. 27, no. 2, pp. 218–228, Mar. 2004. doi: 10.2514/1.9165
    [38]
    P. H. S. Coutinho, M. L. C. Peixoto, I. Bessa, and R. M. Palhares, “Dynamic event-triggered gain-scheduling control of discrete-time quasi-LPV systems,” Automatica, vol. 141, p. 110292, Jul. 2022. doi: 10.1016/j.automatica.2022.110292
    [39]
    H. S. Abbas, R. Tóth, M. Petreczky, N. Meskin, and J. Mohammadpour, “Embedding of nonlinear systems in a linear parameter-varying representation,” IFAC Proc. Vol., vol. 47, no. 3, pp. 6907–6913, 2014. doi: 10.3182/20140824-6-ZA-1003.02506
    [40]
    E. Kofman, J. A. De Doná, and M. M. Seron, “Probabilistic set invariance and ultimate boundedness,” Automatica, vol. 48, no. 10, pp. 2670–2676, Oct. 2012. doi: 10.1016/j.automatica.2012.06.074
    [41]
    S. Mate, H. Kodamana, S. Bhartiya, and P. S. V. Nataraj, “A stabilizing sub-optimal model predictive control for quasi-linear parameter varying systems,” IEEE Control Syst. Lett., vol. 4, no. 2, pp. 402–407, Apr. 2020. doi: 10.1109/LCSYS.2019.2937921
    [42]
    P. Coppens, M. Schuurmans, and P. Patrinos, “Data-driven distributionally robust LQR with multiplicative noise,” in Proc. 2nd Conf. Learning for Dynamics and Control, Berkeley, USA, 2020, pp. 521–530.
    [43]
    R. A. Horn and C. R. Johnson, Matrix Analysis. Cambridge, UK: Cambridge University Press, 2012.
    [44]
    T. Lattimore and C. Szepesvári, Bandit Algorithms. Cambridge, UK: Cambridge University Press, 2020.
    [45]
    M. L. C. Peixoto, M. F. Braga, and R. M. Palhares, “Gain-scheduled control for discrete-time non-linear parameter-varying systems with time-varying delays,” IET Control Theory Appl., vol. 14, no. 19, pp. 3217–3229, Dec. 2020. doi: 10.1049/iet-cta.2020.0900
    [46]
    A. Modares, N. Sadati, B. Esmaeili, F. A. Yaghmaie, and H. Modares, “Safe reinforcement learning via a model-free safety certifier,” IEEE Trans. Neural Netw. Learn. Syst., vol. 35, no. 3, pp. 3302–3311, Mar. 2024. doi: 10.1109/TNNLS.2023.3264815
    [47]
    A. Kwiatkowski, M.-T. Boll, and H. Werner, “Automated generation and assessment of affine LPV models,” in Proc. 45th IEEE Conf. Decision and Control, San Diego, USA, 2006, pp. 6690–6695.

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(13)  / Tables(1)

    Article Metrics

    Article views (57) PDF downloads(21) Cited by()

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return