Risk-Informed Model-Free Safe Control of Linear Parameter-Varying Systems

Babak Esmaeili; Hamidreza Modares

doi:10.1109/JAS.2024.124479

Volume 11 Issue 9

Sep. 2024

IEEE/CAA Journal of Automatica Sinica

JCR Impact Factor: 15.3, Top 1 (SCI Q1)

CiteScore: 23.5, Top 2% (Q1)
Google Scholar h5-index: 77， TOP 5

Turn off MathJax

Article Contents

Article Navigation > IEEE/CAA Journal of Automatica Sinica > 2024 > 11(9): 1918-1932

B. Esmaeili and H. Modares, “Risk-informed model-free safe control of linear parameter-varying systems,” IEEE/CAA J. Autom. Sinica, vol. 11, no. 9, pp. 1918–1932, Sept. 2024. doi: 10.1109/JAS.2024.124479

Citation:

B. Esmaeili and H. Modares, “Risk-informed model-free safe control of linear parameter-varying systems,” IEEE/CAA J. Autom. Sinica, vol. 11, no. 9, pp. 1918–1932, Sept. 2024. doi: 10.1109/JAS.2024.124479

Citation:

PDF( 1591 KB)

Risk-Informed Model-Free Safe Control of Linear Parameter-Varying Systems

doi: 10.1109/JAS.2024.124479

Funds: This work was supported in part by the Department of Navy award (N00014-22-1-2159) and the National Science Foundation under award (ECCS-2227311)

More Information

Author Bio:
Babak Esmaeili (Graduate Student Member, IEEE) received the B.Sc. degree in electrical engineering and control and the M.Sc. degree in mechatronics engineering from the University of Tabriz, Iran, in 2014 and 2017, respectively. He is currently pursuing the Ph.D. degree with the Department of Mechanical Engineering, Michigan State University, USA. His current research interests include data-based control, safe reinforcement learning, and autonomous systems. Mr. Esmaeili serves as a Reviewer for several international journals, including IEEE Transactions on Cybernetics, ISA Transactions, and Journal of Vibration and Control

Hamidreza Modares (Senior Member, IEEE) received the B.S. degree from the University of Tehran, Iran, in 2004, the M.S. degree from the Shahrood University of Technology, Iran, in 2006, and the Ph.D. degree from The University of Texas at Arlington, USA, in 2015. He was a Senior Lecturer with the Shahrood University of Technology from 2006 to 2009 and a Faculty Research Associate with The University of Texas at Arlington from 2015 to 2016. He is currently an Assistant Professor with the Department of Mechanical Engineering, Michigan State University, USA. His current research interests include cyber-physical systems, reinforcement learning, distributed control, robotics, and machine learning. Dr. Modares was a recipient of the Best Paper Award from the 2015 IEEE International Symposium on Resilient Control Systems. He is also an Associate Editor of the IEEE Transactions on Neural Networks and Learning Systems, IEEE Transactions on Systems, Man, and Cybernetics (SMC), and Neurocomputing
Corresponding author: Hamidreza Modares, e-mail: modaresh@msu.edu
¹ To watch the animation of the path tracking performance, please click on the following link: https://youtube.com/shorts/va6sQXzegdw.
² To watch the video of the set-point tracking performance of the robot in Gazebo, please click on the following link: https://www.youtube.com/watch?v=3ULDY4uTluE.
³ To watch the video of the set-point tracking performance of the robot in real-world, please click on the following link: https://www.youtube.com/shorts/uvQIkGUE2fQ.
Received Date: 2024-02-23
Revised Date: 2024-04-04
Accepted Date: 2024-04-14

Available Online: 2024-06-13

Abstract

Abstract

This paper presents a risk-informed data-driven safe control design approach for a class of stochastic uncertain nonlinear discrete-time systems. The nonlinear system is modeled using linear parameter-varying (LPV) systems. A model-based probabilistic safe controller is first designed to guarantee probabilistic $\lambda $-contractivity (i.e., stability and invariance) of the LPV system with respect to a given polyhedral safe set. To obviate the requirement of knowing the LPV system model and to bypass identifying its open-loop model, its closed-loop data-based representation is provided in terms of state and scheduling data as well as a decision variable. It is shown that the variance of the closed-loop system, as well as the probability of safety satisfaction, depends on the decision variable and the noise covariance. A minimum-variance direct data-driven gain-scheduling safe control design approach is presented next by designing the decision variable such that all possible closed-loop system realizations satisfy safety with the highest confidence level. This minimum-variance approach is a control-oriented learning method since it minimizes the variance of the state of the closed-loop system with respect to the safe set, and thus minimizes the risk of safety violation. Unlike the certainty-equivalent approach that results in a risk-neutral control design, the minimum-variance method leads to a risk-averse control design. It is shown that the presented direct risk-averse learning approach requires weaker data richness conditions than existing indirect learning methods based on system identification and can lead to a lower risk of safety violation. Two simulation examples along with an experimental validation on an autonomous vehicle are provided to show the effectiveness of the presented approach.
- Data-driven control,
- linear parameter-varying systems,
- probabilistic control,
- safe control

FullText(HTML)

¹ To watch the animation of the path tracking performance, please click on the following link: https://youtube.com/shorts/va6sQXzegdw.
² To watch the video of the set-point tracking performance of the robot in Gazebo, please click on the following link: https://www.youtube.com/watch?v=3ULDY4uTluE.
³ To watch the video of the set-point tracking performance of the robot in real-world, please click on the following link: https://www.youtube.com/shorts/uvQIkGUE2fQ.

References(47)

References

[1]	L. Brunke, M. Greeff, A. W. Hall, Z. Yuan, S. Zhou, J. Panerati, and A. P. Schoellig, “Safe learning in robotics: From learning-based control to safe reinforcement learning,” Annu. Rev. Control Robot. Auton. Syst., vol. 5, pp. 411–444, May 2022. doi: 10.1146/annurev-control-042920-020211
[2]	M. Zanon and S. Gros, “Safe reinforcement learning using robust MPC,” IEEE Trans. Autom. Control, vol. 66, no. 8, pp. 3638–3652, Aug. 2021. doi: 10.1109/TAC.2020.3024161
[3]	R. Grandia, A. J. Taylor, A. D. Ames, and M. Hutter, “Multi-layered safety for legged robots via control barrier functions and model predictive control,” in Proc. IEEE Int. Conf. Robotics and Automation, Xi'an, China, 2021, pp. 8352-8358.
[4]	M. Mazouchi, S. Nageshrao, and H. Modares, “Conflict-aware safe reinforcement learning: A meta-cognitive learning framework,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 3, pp. 466–481, Mar. 2022. doi: 10.1109/JAS.2021.1004353
[5]	L. Zhang, R. Zhang, T. Wu, R. Weng, M. Han, and Y. Zhao, “Safe reinforcement learning with stability guarantee for motion planning of autonomous vehicles,” IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 12, pp. 5435–5444, Dec. 2021. doi: 10.1109/TNNLS.2021.3084685
[6]	M. Alshiekh, R. Bloem, R. Ehlers, B. Könighofer, S. Niekum, and U. Topcu, “Safe reinforcement learning via shielding,” in Proc. 32nd AAAI Conf. Artificial Intelligence, New Orleans, USA, 2018, pp. 2669–2678.
[7]	S. Li and O. Bastani, “Robust model predictive shielding for safe reinforcement learning with stochastic dynamics,” in Proc. IEEE Int. Conf. Robotics and Automation, Paris, France, 2020, pp. 7166–7172.
[8]	S. Gao, Z. Peng, H. Wang, L. Liu, and D. Wang, “Safety-critical model-free control for multi-target tracking of USVs with collision avoidance,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 7, pp. 1323–1326, Jul. 2022. doi: 10.1109/JAS.2022.105707
[9]	G. Yang, C. Belta, and R. Tron, “Self-triggered control for safety critical systems using control barrier functions,” in Proc. American Control Conf., Philadelphia, USA, 2019, pp. 4454–4459.
[10]	Z. Marvi and B. Kiumarsi, “Barrier-certified learning-enabled safe control design for systems operating in uncertain environments,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 3, pp. 437–449, Mar. 2021.
[11]	M. Ahmadi, X. Xiong, and A. D. Ames, “Risk-averse control via CVaR barrier functions: Application to bipedal robot locomotion,” IEEE Control Syst. Lett., vol. 6, pp. 878–883, 2022. doi: 10.1109/LCSYS.2021.3086854
[12]	R. Cheng, G. Orosz, R. M. Murray, and J. W. Burdick, “End-to-end safe reinforcement learning through barrier functions for safety-critical continuous control tasks,” in Proc. 33rd AAAI Conf. Artificial Intelligence, Honolulu, USA, 2019, pp. 3387–3395.
[13]	J. Zeng, B. Zhang, and K. Sreenath, “Safety-critical model predictive control with discrete-time control barrier function,” in Proc. American Control Conf., New Orleans, USA, 2021, pp. 3882–3889.
[14]	J. Seo, J. Lee, E. Baek, R. Horowitz, and J. Choi, “Safety-critical control with nonaffine control inputs via a relaxed control barrier function for an autonomous vehicle,” IEEE Robot. Autom. Lett., vol. 7, no. 2, pp. 1944–1951, Apr. 2022. doi: 10.1109/LRA.2022.3142408
[15]	A. Chern, X. Wang, A. Iyer, and Y. Nakahira, “Safe control in the presence of stochastic uncertainties,” in Proc. 60th IEEE Conf. Decision and Control, Austin, USA, 2021, pp. 6640–6645.
[16]	A. Agrawal and K. Sreenath, “Discrete control barrier functions for safety-critical control of discrete systems with application to bipedal robot navigation,” in Proc. Robotics: Science and Systems, Cambridge, USA, 2017.
[17]	S. Samuelson and I. Yang, “Safety-aware optimal control of stochastic systems using conditional value-at-risk,” in Proc. Annu. American Control Conf., Milwaukee, USA, 2018, pp. 6285–6290.
[18]	M. F. Reis, A. P. Aguiar, and P. Tabuada, “Control barrier function-based quadratic programs introduce undesirable asymptotically stable equilibria,” IEEE Control Syst. Lett., vol. 5, no. 2, pp. 731–736, Apr. 2021. doi: 10.1109/LCSYS.2020.3004797
[19]	Y. Lan and I. Mezić, “Linearization in the large of nonlinear systems and Koopman operator spectrum,” Phys. D Nonlinear Phenom., vol. 242, no. 1, pp. 42–53, Jan. 2013. doi: 10.1016/j.physd.2012.08.017
[20]	B. Esmaeili, M. Salim, and M. Baradarannia, “Predefined performance-based model-free adaptive fractional-order fast terminal sliding-mode control of MIMO nonlinear systems,” ISA Trans., vol. 131, pp. 108–123, Dec. 2022. doi: 10.1016/j.isatra.2022.05.036
[21]	B. Esmaeili, S. S. Madani, M. Salim, M. Baradarannia, and S. Khanmohammadi, “Model-free adaptive iterative learning integral terminal sliding mode control of exoskeleton robots,” J. Vib. Control, vol. 28, no. 21-22, pp. 3120–3139, Nov. 2022. doi: 10.1177/10775463211026031
[22]	F. Blanchini and S. Miani, Set-Theoretic Methods in Control. Boston, USA: Springer, 2008.
[23]	Z. Gao and J. Fu, “Robust LPV modeling and control of aircraft flying through wind disturbance,” Chin. J. Aeronaut., vol. 32, no. 7, pp. 1588–1602, Jul. 2019. doi: 10.1016/j.cja.2019.03.029
[24]	K. Zhu, D. Ma, and J. Zhao, “Event triggered control for a switched LPV system with applications to aircraft engines,” IET Control Theory Appl., vol. 12, no. 10, pp. 1505–1514, Jul. 2018. doi: 10.1049/iet-cta.2017.0895
[25]	A. San-Miguel, V. Puig, and G. Alenyà, “Disturbance observer-based LPV feedback control of a N-DoF robotic manipulator including compliance through gain shifting,” Control Eng. Pract., vol. 115, p. 104887, Oct. 2021. doi: 10.1016/j.conengprac.2021.104887
[26]	P. S. G. Cisneros, A. Sridharan, and H. Werner, “Constrained predictive control of a robotic manipulator using quasi-LPV representations,” IFAC-PapersOnLine, vol. 51, no. 26, pp. 118–123, 2018. doi: 10.1016/j.ifacol.2018.11.158
[27]	M. A. H. Darwish, P. B. Cox, I. Proimadis, G. Pillonetto, and R. Tóth, “Prediction-error identification of LPV systems: A nonparametric Gaussian regression approach,” Automatica, vol. 97, pp. 92–103, Nov. 2018. doi: 10.1016/j.automatica.2018.07.032
[28]	A. Bisoffi, C. De Persis, and P. Tesi, “Data-based guarantees of set invariance properties,” IFAC-PapersOnLine, vol. 53, no. 2, pp. 3953–3958, 2020. doi: 10.1016/j.ifacol.2020.12.2250
[29]	A. Luppi, C. De Persis, and P. Tesi, “On data-driven stabilization of systems with nonlinearities satisfying quadratic constraints,” Syst. Control Lett., vol. 163, p. 105206, May 2022. doi: 10.1016/j.sysconle.2022.105206
[30]	A. Bisoffi, C. De Persis, and P. Tesi, “Controller design for robust invariance from noisy data,” IEEE Trans. Autom. Control, vol. 68, no. 1, pp. 636–643, Jan. 2023. doi: 10.1109/TAC.2022.3170373
[31]	C. De Persis and P. Tesi, “Low-complexity learning of linear quadratic regulators from noisy data,” Automatica, vol. 128, p. 109548, Jun. 2021. doi: 10.1016/j.automatica.2021.109548
[32]	H. Modares, “Data-driven safe control of uncertain linear systems under aleatory uncertainty,” IEEE Trans. Autom. Control, vol. 69, no. 1, pp. 551–558, Jan. 2024. doi: 10.1109/TAC.2023.3267019
[33]	J. M. Snider, “Automatic steering methods for autonomous automobile path tracking,” Carnegie Mellon University, Pittsburgh, USA, Tech. Rep. CMU-RI-TR-09-08, 2009.
[34]	E. Alcala, V. Puig, J. Quevedo, and T. Escobet, “Gain-scheduling LPV control for autonomous vehicles including friction force estimation and compensation mechanism,” IET Control Theory Appl., vol. 12, no. 12, pp. 1683–1693, Aug. 2018. doi: 10.1049/iet-cta.2017.1154
[35]	E. Alcalá, V. Puig, and J. Quevedo, “LPV-MPC control for autonomous vehicles,” IFAC-PapersOnLine, vol. 52, no. 28, pp. 106–113, 2019. doi: 10.1016/j.ifacol.2019.12.356
[36]	X. Geng and L. Xie, “Data-driven decision making in power systems with probabilistic guarantees: Theory and applications of chance-constrained optimization,” Annu. Rev. Control, vol. 47, pp. 341–363, 2019. doi: 10.1016/j.arcontrol.2019.05.005
[37]	A. Marcos and G. J. Balas, “Development of linear-parameter-varying models for aircraft,” J. Guid. Control Dyn., vol. 27, no. 2, pp. 218–228, Mar. 2004. doi: 10.2514/1.9165
[38]	P. H. S. Coutinho, M. L. C. Peixoto, I. Bessa, and R. M. Palhares, “Dynamic event-triggered gain-scheduling control of discrete-time quasi-LPV systems,” Automatica, vol. 141, p. 110292, Jul. 2022. doi: 10.1016/j.automatica.2022.110292
[39]	H. S. Abbas, R. Tóth, M. Petreczky, N. Meskin, and J. Mohammadpour, “Embedding of nonlinear systems in a linear parameter-varying representation,” IFAC Proc. Vol., vol. 47, no. 3, pp. 6907–6913, 2014. doi: 10.3182/20140824-6-ZA-1003.02506
[40]	E. Kofman, J. A. De Doná, and M. M. Seron, “Probabilistic set invariance and ultimate boundedness,” Automatica, vol. 48, no. 10, pp. 2670–2676, Oct. 2012. doi: 10.1016/j.automatica.2012.06.074
[41]	S. Mate, H. Kodamana, S. Bhartiya, and P. S. V. Nataraj, “A stabilizing sub-optimal model predictive control for quasi-linear parameter varying systems,” IEEE Control Syst. Lett., vol. 4, no. 2, pp. 402–407, Apr. 2020. doi: 10.1109/LCSYS.2019.2937921
[42]	P. Coppens, M. Schuurmans, and P. Patrinos, “Data-driven distributionally robust LQR with multiplicative noise,” in Proc. 2nd Conf. Learning for Dynamics and Control, Berkeley, USA, 2020, pp. 521–530.
[43]	R. A. Horn and C. R. Johnson, Matrix Analysis. Cambridge, UK: Cambridge University Press, 2012.
[44]	T. Lattimore and C. Szepesvári, Bandit Algorithms. Cambridge, UK: Cambridge University Press, 2020.
[45]	M. L. C. Peixoto, M. F. Braga, and R. M. Palhares, “Gain-scheduled control for discrete-time non-linear parameter-varying systems with time-varying delays,” IET Control Theory Appl., vol. 14, no. 19, pp. 3217–3229, Dec. 2020. doi: 10.1049/iet-cta.2020.0900
[46]	A. Modares, N. Sadati, B. Esmaeili, F. A. Yaghmaie, and H. Modares, “Safe reinforcement learning via a model-free safety certifier,” IEEE Trans. Neural Netw. Learn. Syst., vol. 35, no. 3, pp. 3302–3311, Mar. 2024. doi: 10.1109/TNNLS.2023.3264815
[47]	A. Kwiatkowski, M.-T. Boll, and H. Werner, “Automated generation and assessment of affine LPV models,” in Proc. 45th IEEE Conf. Decision and Control, San Diego, USA, 2006, pp. 6690–6695.

Supplements(0)

Cited By

Proportional views

Proportional views

通讯作者: 陈斌, bchen63@163.com

1.
沈阳化工大学材料科学与工程学院沈阳 110142

Figures(13) / Tables(1)

Get Citation

PDF

XML

Article Metrics

Article views (244) PDF downloads(69)

Highlights

Data-Driven Safe Control: Introduces a method for designing safe controls in nonlinear systems using direct data, without requiring system models
Risk-Averse Design: Implements a minimum-variance approach to enhance safety and reduce risk, favoring robust over neutral strategies
Practical Validation: Demonstrates effectiveness through simulations and an experimental autonomous vehicle application

Risk-Informed Model-Free Safe Control of Linear Parameter-Varying Systems

doi: 10.1109/JAS.2024.124479

Abstract

References

Proportional views

Catalog

通讯作者: 陈斌, bchen63@163.com

Article Metrics

Highlights

Export File

Citation

Format

Content