A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 15.3, Top 1 (SCI Q1)
    CiteScore: 23.5, Top 2% (Q1)
    Google Scholar h5-index: 77, TOP 5
Turn off MathJax
Article Contents
B. Esmaeili and H. Modares, “Risk-informed model-free safe control of linear parameter-varying systems,” IEEE/CAA J. Autom. Sinica, vol. 11, no. 9, pp. 1–15, Sept. 2024. doi: 10.1109/JAS.2024.124479
Citation: B. Esmaeili and H. Modares, “Risk-informed model-free safe control of linear parameter-varying systems,” IEEE/CAA J. Autom. Sinica, vol. 11, no. 9, pp. 1–15, Sept. 2024. doi: 10.1109/JAS.2024.124479

Risk-Informed Model-Free Safe Control of Linear Parameter-Varying Systems

doi: 10.1109/JAS.2024.124479
Funds:  This work was supported in part by the Department of Navy award (N00014-22-1-2159) and the National Science Foundation under award (ECCS-2227311)
More Information
  • This paper presents a risk-informed data-driven safe control design approach for a class of stochastic uncertain nonlinear discrete-time systems. The nonlinear system is modeled using linear parameter-varying (LPV) systems. A model-based probabilistic safe controller is first designed to guarantee probabilistic $\lambda$-contractivity (i.e., stability and invariance) of the LPV system with respect to a given polyhedral safe set. To obviate the requirement of knowing the LPV system model and to bypass identifying its open-loop model, its closed-loop data-based representation is provided in terms of state and scheduling data as well as a decision variable. It is shown that the variance of the closed-loop system, as well as the probability of safety satisfaction, depends on the decision variable and the noise covariance. A minimum-variance direct data-driven gain-scheduling safe control design approach is presented next by designing the decision variable such that all possible closed-loop system realizations satisfy safety with the highest confidence level. This minimum-variance approach is a control-oriented learning method since it minimizes the variance of the state of the closed-loop system with respect to the safe set, and thus minimizes the risk of safety violation. Unlike the certainty-equivalent approach that results in a risk-neutral control design, the minimum-variance method leads to a risk-averse control design. It is shown that the presented direct risk-averse learning approach requires weaker data richness conditions than existing indirect learning methods based on system identification and can lead to a lower risk of safety violation. Two simulation examples along with an experimental validation on an autonomous vehicle are provided to show the effectiveness of the presented approach.


  • loading
  • 1 To watch the animation of the path tracking performance, please click on the following link: Safe path tracking animation.
    2 To watch the video of the set-point tracking performance of the robot in Gazebo, please click on the following link: Safe set-point tracking simulation.
    3 To watch the video of the set-point tracking performance of the robot in real-world, please click on the following link: Safe set-point tracking implementation.
  • [1]
    L. Brunke, M. Greeff, A. W. Hall, Z. Yuan, S. Zhou, J. Panerati, and A. P. Schoellig, “Safe learning in robotics: From learning-based control to safe reinforcement learning,” Annu. Rev. Control Robot. Auton. Syst., vol. 5, pp. 411–444, May 2022. doi: 10.1146/annurev-control-042920-020211
    M. Zanon and S. Gros, “Safe reinforcement learning using robust MPC,” IEEE Trans. Autom. Control, vol. 66, no. 8, pp. 3638–3652, Aug. 2021. doi: 10.1109/TAC.2020.3024161
    R. Grandia, A. J. Taylor, A. D. Ames, and M. Hutter, “Multi-layered safety for legged robots via control barrier functions and model predictive control,” in Proc. IEEE Int. Conf. Robotics and Automation, Xi'an, China, 2021, pp. 8352-8358.
    M. Mazouchi, S. Nageshrao, and H. Modares, “Conflict-aware safe reinforcement learning: A meta-cognitive learning framework,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 3, pp. 466–481, Mar. 2022. doi: 10.1109/JAS.2021.1004353
    L. Zhang, R. Zhang, T. Wu, R. Weng, M. Han, and Y. Zhao, “Safe reinforcement learning with stability guarantee for motion planning of autonomous vehicles,” IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 12, pp. 5435–5444, Dec. 2021. doi: 10.1109/TNNLS.2021.3084685
    M. Alshiekh, R. Bloem, R. Ehlers, B. Könighofer, S. Niekum, and U. Topcu, “Safe reinforcement learning via shielding,” in Proc. 32nd AAAI Conf. Artificial Intelligence, New Orleans, USA, 2018, pp. 2669–2678.
    S. Li and O. Bastani, “Robust model predictive shielding for safe reinforcement learning with stochastic dynamics,” in Proc. IEEE Int. Conf. Robotics and Automation, Paris, France, 2020, pp. 7166–7172.
    S. Gao, Z. Peng, H. Wang, L. Liu, and D. Wang, “Safety-critical model-free control for multi-target tracking of USVs with collision avoidance,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 7, pp. 1323–1326, Jul. 2022. doi: 10.1109/JAS.2022.105707
    G. Yang, C. Belta, and R. Tron, “Self-triggered control for safety critical systems using control barrier functions,” in Proc. American Control Conf., Philadelphia, USA, 2019, pp. 4454–4459.
    Z. Marvi and B. Kiumarsi, “Barrier-certified learning-enabled safe control design for systems operating in uncertain environments,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 3, pp. 437–449, Mar. 2021.
    M. Ahmadi, X. Xiong, and A. D. Ames, “Risk-averse control via CVaR barrier functions: Application to bipedal robot locomotion,” IEEE Control Syst. Lett., vol. 6, pp. 878–883, 2022. doi: 10.1109/LCSYS.2021.3086854
    R. Cheng, G. Orosz, R. M. Murray, and J. W. Burdick, “End-to-end safe reinforcement learning through barrier functions for safety-critical continuous control tasks,” in Proc. 33rd AAAI Conf. Artificial Intelligence, Honolulu, USA, 2019, pp. 3387–3395.
    J. Zeng, B. Zhang, and K. Sreenath, “Safety-critical model predictive control with discrete-time control barrier function,” in Proc. American Control Conf., New Orleans, USA, 2021, pp. 3882–3889.
    J. Seo, J. Lee, E. Baek, R. Horowitz, and J. Choi, “Safety-critical control with nonaffine control inputs via a relaxed control barrier function for an autonomous vehicle,” IEEE Robot. Autom. Lett., vol. 7, no. 2, pp. 1944–1951, Apr. 2022. doi: 10.1109/LRA.2022.3142408
    A. Chern, X. Wang, A. Iyer, and Y. Nakahira, “Safe control in the presence of stochastic uncertainties,” in Proc. 60th IEEE Conf. Decision and Control, Austin, USA, 2021, pp. 6640–6645.
    A. Agrawal and K. Sreenath, “Discrete control barrier functions for safety-critical control of discrete systems with application to bipedal robot navigation,” in Proc. Robotics: Science and Systems, Cambridge, USA, 2017.
    S. Samuelson and I. Yang, “Safety-aware optimal control of stochastic systems using conditional value-at-risk,” in Proc. Annu. American Control Conf., Milwaukee, USA, 2018, pp. 6285–6290.
    M. F. Reis, A. P. Aguiar, and P. Tabuada, “Control barrier function-based quadratic programs introduce undesirable asymptotically stable equilibria,” IEEE Control Syst. Lett., vol. 5, no. 2, pp. 731–736, Apr. 2021. doi: 10.1109/LCSYS.2020.3004797
    Y. Lan and I. Mezić, “Linearization in the large of nonlinear systems and Koopman operator spectrum,” Phys. D Nonlinear Phenom., vol. 242, no. 1, pp. 42–53, Jan. 2013. doi: 10.1016/j.physd.2012.08.017
    B. Esmaeili, M. Salim, and M. Baradarannia, “Predefined performance-based model-free adaptive fractional-order fast terminal sliding-mode control of MIMO nonlinear systems,” ISA Trans., vol. 131, pp. 108–123, Dec. 2022. doi: 10.1016/j.isatra.2022.05.036
    B. Esmaeili, S. S. Madani, M. Salim, M. Baradarannia, and S. Khanmohammadi, “Model-free adaptive iterative learning integral terminal sliding mode control of exoskeleton robots,” J. Vib. Control, vol. 28, no. 21-22, pp. 3120–3139, Nov. 2022. doi: 10.1177/10775463211026031
    F. Blanchini and S. Miani, Set-Theoretic Methods in Control. Boston, USA: Springer, 2008.
    Z. Gao and J. Fu, “Robust LPV modeling and control of aircraft flying through wind disturbance,” Chin. J. Aeronaut., vol. 32, no. 7, pp. 1588–1602, Jul. 2019. doi: 10.1016/j.cja.2019.03.029
    K. Zhu, D. Ma, and J. Zhao, “Event triggered control for a switched LPV system with applications to aircraft engines,” IET Control Theory Appl., vol. 12, no. 10, pp. 1505–1514, Jul. 2018. doi: 10.1049/iet-cta.2017.0895
    A. San-Miguel, V. Puig, and G. Alenyà, “Disturbance observer-based LPV feedback control of a N-DoF robotic manipulator including compliance through gain shifting,” Control Eng. Pract., vol. 115, p. 104887, Oct. 2021. doi: 10.1016/j.conengprac.2021.104887
    P. S. G. Cisneros, A. Sridharan, and H. Werner, “Constrained predictive control of a robotic manipulator using quasi-LPV representations,” IFAC-PapersOnLine, vol. 51, no. 26, pp. 118–123, 2018. doi: 10.1016/j.ifacol.2018.11.158
    M. A. H. Darwish, P. B. Cox, I. Proimadis, G. Pillonetto, and R. Tóth, “Prediction-error identification of LPV systems: A nonparametric Gaussian regression approach,” Automatica, vol. 97, pp. 92–103, Nov. 2018. doi: 10.1016/j.automatica.2018.07.032
    A. Bisoffi, C. De Persis, and P. Tesi, “Data-based guarantees of set invariance properties,” IFAC-PapersOnLine, vol. 53, no. 2, pp. 3953–3958, 2020. doi: 10.1016/j.ifacol.2020.12.2250
    A. Luppi, C. De Persis, and P. Tesi, “On data-driven stabilization of systems with nonlinearities satisfying quadratic constraints,” Syst. Control Lett., vol. 163, p. 105206, May 2022. doi: 10.1016/j.sysconle.2022.105206
    A. Bisoffi, C. De Persis, and P. Tesi, “Controller design for robust invariance from noisy data,” IEEE Trans. Autom. Control, vol. 68, no. 1, pp. 636–643, Jan. 2023. doi: 10.1109/TAC.2022.3170373
    C. De Persis and P. Tesi, “Low-complexity learning of linear quadratic regulators from noisy data,” Automatica, vol. 128, p. 109548, Jun. 2021. doi: 10.1016/j.automatica.2021.109548
    H. Modares, “Data-driven safe control of uncertain linear systems under aleatory uncertainty,” IEEE Trans. Autom. Control, vol. 69, no. 1, pp. 551–558, Jan. 2024. doi: 10.1109/TAC.2023.3267019
    J. M. Snider, “Automatic steering methods for autonomous automobile path tracking,” Carnegie Mellon University, Pittsburgh, USA, Tech. Rep. CMU-RI-TR-09-08, 2009.
    E. Alcala, V. Puig, J. Quevedo, and T. Escobet, “Gain-scheduling LPV control for autonomous vehicles including friction force estimation and compensation mechanism,” IET Control Theory Appl., vol. 12, no. 12, pp. 1683–1693, Aug. 2018. doi: 10.1049/iet-cta.2017.1154
    E. Alcalá, V. Puig, and J. Quevedo, “LPV-MPC control for autonomous vehicles,” IFAC-PapersOnLine, vol. 52, no. 28, pp. 106–113, 2019. doi: 10.1016/j.ifacol.2019.12.356
    X. Geng and L. Xie, “Data-driven decision making in power systems with probabilistic guarantees: Theory and applications of chance-constrained optimization,” Annu. Rev. Control, vol. 47, pp. 341–363, 2019. doi: 10.1016/j.arcontrol.2019.05.005
    A. Marcos and G. J. Balas, “Development of linear-parameter-varying models for aircraft,” J. Guid. Control Dyn., vol. 27, no. 2, pp. 218–228, Mar. 2004. doi: 10.2514/1.9165
    P. H. S. Coutinho, M. L. C. Peixoto, I. Bessa, and R. M. Palhares, “Dynamic event-triggered gain-scheduling control of discrete-time quasi-LPV systems,” Automatica, vol. 141, p. 110292, Jul. 2022. doi: 10.1016/j.automatica.2022.110292
    H. S. Abbas, R. Tóth, M. Petreczky, N. Meskin, and J. Mohammadpour, “Embedding of nonlinear systems in a linear parameter-varying representation,” IFAC Proc. Vol., vol. 47, no. 3, pp. 6907–6913, 2014. doi: 10.3182/20140824-6-ZA-1003.02506
    E. Kofman, J. A. De Doná, and M. M. Seron, “Probabilistic set invariance and ultimate boundedness,” Automatica, vol. 48, no. 10, pp. 2670–2676, Oct. 2012. doi: 10.1016/j.automatica.2012.06.074
    S. Mate, H. Kodamana, S. Bhartiya, and P. S. V. Nataraj, “A stabilizing sub-optimal model predictive control for quasi-linear parameter varying systems,” IEEE Control Syst. Lett., vol. 4, no. 2, pp. 402–407, Apr. 2020. doi: 10.1109/LCSYS.2019.2937921
    P. Coppens, M. Schuurmans, and P. Patrinos, “Data-driven distributionally robust LQR with multiplicative noise,” in Proc. 2nd Conf. Learning for Dynamics and Control, Berkeley, USA, 2020, pp. 521–530.
    R. A. Horn and C. R. Johnson, Matrix Analysis. Cambridge, UK: Cambridge University Press, 2012.
    T. Lattimore and C. Szepesvári, Bandit Algorithms. Cambridge, UK: Cambridge University Press, 2020.
    M. L. C. Peixoto, M. F. Braga, and R. M. Palhares, “Gain-scheduled control for discrete-time non-linear parameter-varying systems with time-varying delays,” IET Control Theory Appl., vol. 14, no. 19, pp. 3217–3229, Dec. 2020. doi: 10.1049/iet-cta.2020.0900
    A. Modares, N. Sadati, B. Esmaeili, F. A. Yaghmaie, and H. Modares, “Safe reinforcement learning via a model-free safety certifier,” IEEE Trans. Neural Netw. Learn. Syst., vol. 35, no. 3, pp. 3302–3311, Mar. 2024. doi: 10.1109/TNNLS.2023.3264815
    A. Kwiatkowski, M.-T. Boll, and H. Werner, “Automated generation and assessment of affine LPV models,” in Proc. 45th IEEE Conf. Decision and Control, San Diego, USA, 2006, pp. 6690–6695.


    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(13)  / Tables(1)

    Article Metrics

    Article views (57) PDF downloads(21) Cited by()


    DownLoad:  Full-Size Img  PowerPoint