A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Volume 5 Issue 5
Aug.  2018

IEEE/CAA Journal of Automatica Sinica

• JCR Impact Factor: 11.8, Top 4% (SCI Q1)
CiteScore: 17.6, Top 3% (Q1)
Google Scholar h5-index: 77， TOP 5
Turn off MathJax
Article Contents
Xuesong Chen and Xin Chen, "An Iterative Method for Optimal Feedback Control and Generalized HJB Equation," IEEE/CAA J. Autom. Sinica, vol. 5, no. 5, pp. 999-1006, Sept. 2018. doi: 10.1109/JAS.2017.7510706
 Citation: Xuesong Chen and Xin Chen, "An Iterative Method for Optimal Feedback Control and Generalized HJB Equation," IEEE/CAA J. Autom. Sinica, vol. 5, no. 5, pp. 999-1006, Sept. 2018.

# An Iterative Method for Optimal Feedback Control and Generalized HJB Equation

##### doi: 10.1109/JAS.2017.7510706
Funds:

the National Natural Science Foundation of China U1601202

the National Natural Science Foundation of China U1134004

the National Natural Science Foundation of China 91648108

the Natural Science Foundation of Guangdong Province 2015A030313497

the Natural Science Foundation of Guangdong Province 2015A030312008

the Project of Science and Technology of Guangdong Province 2015B010102014

the Project of Science and Technology of Guangdong Province 2015B010124001

the Project of Science and Technology of Guangdong Province 2015B010104006

the Project of Science and Technology of Guangdong Province 2018A030313505

• In this paper, a new iterative method is proposed to solve the generalized Hamilton-Jacobi-Bellman (GHJB) equation through successively approximate it. Firstly, the GHJB equation is converted to an algebraic equation with the vector norm, which is essentially a set of simultaneous nonlinear equations in the case of dynamic systems. Then, the proposed algorithm solves GHJB equation numerically for points near the origin by considering the linearization of the non-linear equations under a good initial control guess. Finally, the procedure is proved to converge to the optimal stabilizing solution with respect to the iteration variable. In addition, it is shown that the result is a closed-loop control based on this iterative approach. Illustrative examples show that the update control laws will converge to optimal control for nonlinear systems.

•  [1] D. E. Kirk, Optimal Control Theory: An Introduction. Mineola, USA: Courier Corporation, 2012. [2] F. L. Lewis and V. L. Syrmos, Optimal Control. New York, USA: Wiley, 1995. [3] R. Bellman, Dynamic Programming. New Jersey, USA: Princeton University Press, 1957. [4] D. Kleinman, "On an iterative technique for riccati equation computations, " IEEE Trans. Autom. Contr., vol. 13, no. 1, pp. 114-115, Feb. 1968. http://xueshu.baidu.com/s?wd=paperuri%3A%28c172df92b76abf7a3551f8ab95408337%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fci.nii.ac.jp%2Fnaid%2F30019918845&ie=utf-8&sc_us=16210463245255859084 [5] F. Y. Wang, H. G. Zhang, and D. R. Liu, "Adaptive dynamic programming: An introduction, " IEEE Comput. Intell. Mag., vol. 4, no. 2, pp. 39-47, May 2009. http://xueshu.baidu.com/s?wd=paperuri%3A%282a885ce7daacb3621170ec72c98d3cf9%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Ficp.jsp%3Farnumber%3D4840325&ie=utf-8&sc_us=17790335501086901414 [6] F. L. Lewis, D. Vrabie, and K. G. Vamvoudakis, "Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers, " IEEE Control Syst., vol. 32, no. 6, pp. 76-105, Dec. 2012. http://xueshu.baidu.com/s?wd=paperuri%3A%2860ee797c5b81e82150cf13d045f5fc09%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Ficp.jsp%3Farnumber%3D6315769&ie=utf-8&sc_us=15413527564144328925 [7] W. Zhang, W. X. Liu, X. Wang, L. M. Liu, and F. Ferrese, "Online optimal generation control based on constrained distributed gradient algorithm, " IEEE Trans. Power Syst., vol. 30, no. 1, pp. 35-45, Jan. 2015. http://xueshu.baidu.com/s?wd=paperuri%3A%28d8e857a156c4bc17f4164e996d00b13a%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fdx.doi.org%2F10.1109%2FTPWRS.2014.2319315&ie=utf-8&sc_us=4468568846486416817 [8] N. Sakamoto and A. J. van der Schaft, "Analytical approximation methods for the stabilizing solution of the Hamilton-Jacobi equation, " IEEE Trans. Autom. Control, vol. 53, no. 10, pp. 2335-2350, Nov. 2008. http://www.mendeley.com/research/analytical-approximation-methods-stabilizing-solution-hamiltonjacobi-equation/ [9] T. Bian, Y. Jiang, and Z. P. Jiang, "Adaptive dynamic programming and optimal control of nonlinear nonaffine systems, " Automatica, vol. 50, no. 10, pp. 2624-2632, Oct. 2014. http://xueshu.baidu.com/s?wd=paperuri%3A%287c5a4b93543a6cceb05ac88bb1e68464%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fdialnet.unirioja.es%2Fservlet%2Farticulo%3Fcodigo%3D4854670&ie=utf-8&sc_us=3034413140015552710 [10] C. O. Aguilar and A. J. Krener, "Numerical solutions to the Bellman equation of optimal control, " J. Optim. Theory Appl., vol. 160, no. 2, pp. 527-552, Feb. 2014. [11] S. Cacace, E. Cristiani, M. Falcone, and A. Picarelli, "A patchy dynamic programming scheme for a class of Hamilton-Jacobi-Bellman equations, " SIAM J. Sci. Comput., vol. 34, no. 5, pp. A2625-A2649, Oct. 2012. http://xueshu.baidu.com/s?wd=paperuri%3A%286e34c72862da9ed0180ea341f0816b0c%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fadsabs.harvard.edu%2Fabs%2F2011arXiv1109.3577C&ie=utf-8&sc_us=9863127530291330464 [12] N. Govindarajan, C. C. de Visser, and K. Krishnakumar, "A sparse collocation method for solving time-dependent HJB equations using multivariate B-splines, " Automatica, vol. 50, no. 9, pp. 2234-2244, Sep. 2014. http://xueshu.baidu.com/s?wd=paperuri%3A%28232ae8ead0425f554db5acf7747afd31%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fdialnet.unirioja.es%2Fservlet%2Farticulo%3Fcodigo%3D4830522&ie=utf-8&sc_us=2303783704440044212 [13] J. Markman and I. N. Katz, "An iterative algorithm for solving Hamilton-Jacobi type equations, " SIAM J. Sci. Comput., vol. 22, no. 1, pp. 312-329, Jul. 2000. [14] I. Smears and E. Süli, "Discontinuous Galerkin finite element approximation of Hamilton-Jacobi-Bellman equations with Cordes coefficients, "SIAM J. Numer. Anal., vol. 52, no. 2, pp. 993-1016, Apr. 2014. http://eprints.maths.ox.ac.uk/1671 [15] R. W. Beard, G. N. Saridis, and J. T. Wen, "Approximate solutions to the time-invariant Hamilton-Jacobi-Bellman equation, " J. Optim. Theory Appl., vol. 96, no. 3, pp. 589-626, Mar. 1998. http://xueshu.baidu.com/s?wd=paperuri%3A%28e806b8acd4f6c5306a9c2870839fa064%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fci.nii.ac.jp%2Fnaid%2F80010226858&ie=utf-8&sc_us=15790908467176106295 [16] R. W. Beard, G. N. Saridis, and J. T. Wen, "Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation, " Automatica, vol. 33, no. 12, pp. 2159-2177, Dec. 1997. http://xueshu.baidu.com/s?wd=paperuri%3A%281e50ce49db952defceb5de4d82886749%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fdl.acm.org%2Fcitation.cfm%3Fid%3D280068%26amp%3Bpreflayout%3Dflat&ie=utf-8&sc_us=11738833470501958577 [17] G. N. Saridis and C. S. G. Lee, "An approximation theory of optimal control for trainable manipulators, " IEEE Trans. Syst. Man Cybern. vol. 9, no. 3, pp. 152-159, Mar. 1979. http://xueshu.baidu.com/s?wd=paperuri%3A%28fafa925f04afc7b2d4167eec6f3a33b4%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fci.nii.ac.jp%2Fnaid%2F80013924674&ie=utf-8&sc_us=1501191005870337592 [18] Q. Gong, W. Kang, and I. M. Ross, "A pseudospectral method for the optimal control of constrained feedback linearizable systems, " IEEE Trans. Autom. Control, vol. 51, no. 7, pp. 1115-1129, Jul. 2006. http://www.mendeley.com/catalog/pseudospectral-method-optimal-control-constrained-feedback-linearizable-systems/

### Catalog

###### 通讯作者: 陈斌, bchen63@163.com
• 1.

沈阳化工大学材料科学与工程学院 沈阳 110142

Figures(5)  / Tables(2)

## Article Metrics

Article views (1066) PDF downloads(29) Cited by()

/