I. INTRODUCTION

The study of multi-agent coordination was perhaps first motivated by the group phenomena in nature^[1],the work in statistical physics^[2] and distributed computing^[3]. Since that,in particular for the past two decades,cooperative control of multi-agent systems has received increasing attention by the fact that many benefits can be obtained when a single complicated agent is equivalently replaced by multiple simpler agents. And in the recent ten years,numerous results have been obtained to solve a variety of multi-agent cooperative control problems. In early works (before and around the year 2000),most of them assumed the availability of global team information,the ability to plan group actions in a centralized manner,and/or perfect and unlimited communication among the agents; see ^{[4, 5, 6, 7, 8, 9]}. However,the centralized coordination schemes do not work well for the reason that the real-world communication topologies are usually not fully connected,especially when the group of agents have large scale members. Due to the disadvantage of the centralized coordination schemes,the distributed manner was investigated. In the distributed control of a group of autonomous agents,the main objective is typically to have the whole group of agents working in a cooperative fashion through a distributed protocol. These controllers are distributed in the sense that the controller design for each agent only requires relative state information between itself and its neighbors. Hence,cooperative or coordination refers to a close relationship among all agents in the group where information sharing plays a crucial role. The distributed approach has many advantages for achieving cooperative group performance,especially with low operational costs,less system requirements,high robustness,strong adaptivity,and flexible scalability. The control theory of multi-agent systems can be applied to many practical engineering applications such as cooperative control of unmanned ground/air/underwater vehicles^{[10, 11, 12]},distributed sensor networks^{[13, 14]}, aggregation and rendezvous control^[15],attitude alignment of spacecraft^{[16, 17]} and so on. Therefore,cooperative control of multi-agent systems has been widely recognized and appreciated, and numerous results have been obtained to solve multi-agent cooperative control problems. There are several reviews on multi-agent systems given in ^{[18, 19, 20, 21, 22, 23, 24, 25]} and the classic books in ^{[26, 27, 28, 29, 30, 31, 32, 33]}.

Among the existing works mentioned above,most of them studied only the first- and second-order dynamics. Recently,some researchers turned to focus on the high-order multi-agent systems coordination. One motivation for studying higher-order consensus comes from observing the behavior of flocks of birds^[34]. It is often noted that such flocks fly in formation,maintaining a nominal separation from each other,but each traveling with the same velocity vector. It can be shown that second-order coordination can reproduce the behavior of a separation and common velocity under information exchange. However,sometimes a bird flock abruptly changes direction,perhaps when one of them suddenly perceives a source of danger or food. Clearly,the birds in this setting need to build consensus on not only their relative position and velocity, but also on their acceleration. In addition,in engineering,many systems are modeled by higher-order dynamics. For example,the jerk systems,described by third-order differential equations,are of particular interest in mechanical engineering^[35]. A single link flexible joint manipulator is well modeled by a fourth-order nonlinear system^[36]. To sum up,in many practical applications,it is not acceptable to model the plant dynamics with only single- or double-integrator dynamics. So it is necessary to extent the coordination problem from lower-order dynamics to high-order ones.

This paper reviews the major results and recent progress in distributed high-order linear and nonlinear multi-agent coordination. Specifically,this paper reviews the research results in the following directions,which are not independent but actually may have overlapping to some extent. In Section II,the necessary preliminaries concerning the cooperative control problem are given, especially the graph theory. In Section III,the coordination of high-order linear multi-agent systems are considered. First,the typical and basic model of the high-order linear multi-agent systems is investigated. Second,coordination of high-order linear systems with disturbance are discussed. The remaining subsections describe the coordination of discrete-time and general high-order linear time-invariant (LTI) systems. In Section IV,the coordination of high-order nonlinear multi-agent systems is investigated,including the typical high-order nonlinear system,high-order nonholonomic systems and nonlinear systems in the Brunovsky form. The last section gives a short discussion to summarize the paper and propose several promising research directions along with some open problems that are deemed important for further investigations.

II. PRELIMINARIES

Graph theory is a very useful mathematical tool in the research of coordination of multi-agent systems. When a team of $N$ agents labeled as systems $1$ to $N$ are considered,the topology of a communication network can be expressed by a graph,either directed or undirected,according to whether the information flow is unidirectional or bidirectional.

A weighted graph (undirected graph or digraph) $\mathcal{G} =(\mathcal{V}$,$\mathcal{E},\mathcal{A})$ consists of a nonempty finite set of $N$ nodes $\mathcal{V} =$ $\{v_1,v_2,\cdots,v_N\}$, a set of edges $\mathcal{E} \subseteq \mathcal{V} \times \mathcal{V}$,and a weighted adjacency matrix $\mathcal{A}=[a_{ij}] \in {\bf R}^{N\times N}$,where $\mathcal{V}$ is a set of the indices of the systems and $\mathcal{E}$ is a set of edges that describe the communication between the agents. In a directed graph, if $(v_i,v_j)\in \mathcal{E}$,then $i$ is neighboring to $j$, meaning system $j$ can obtain information from system $i$. And it can be said that $v_i$ is the parent node and $v_j$ is the child node. The set of neighbors of node $i$ is denoted as $N_i =\{j|(v_j, v_i) \in \mathcal{E}\}$. In most existing literature,self edges $(v_i,v_i)$ are assumed to be not allowed. Each entry $a_{ij}$ of the adjacency matrix $\mathcal{A}$ is the weight associated with edge $(v_j,v_i)$ and $a_{ij} > 0$ if $(v_j,v_i)\in\mathcal{E}$. Otherwise,$a_{ij} = 0$. For an undirected graph $\mathcal{G}$,the weighted adjacency matrix $\mathcal{A}=[a_{ij}]_{N\times N}$ satisfies $a_{ij}$ $=$ $a_{ji}$ $>$ $0$. Its Laplacian matrix $L=[L_{ij}]$ is defined as

\begin{align} L_{ij}=\left\{\begin{array}{ll} -a_{ij},& \mathrm{if}\ i\neq j\ \mathrm{and}\ i\in\mathcal{N}_{j},\\[2mm] \sum\limits_{l\in\mathcal{N}_{j}}a_{jl},& \mathrm{if}\ i=j ,\\[2mm] 0,& \mathrm{otherwise}. \end{array} \right. \end{align}

(1)

Define the in-degree of node $i$ as $d_i =\sum_{j=1}^Na_{ij}$ and the in-degree matrix as $D = {\rm diag}\{d_i\} \in {\bf R}^{N\times N}$. Then,the graph Laplacian matrix is $L = D-A$. Let $1_N =$ $[1$,$1$,$\cdots$,$1]^{\rm T} \in {\bf R}^N$; then,$L_{1_N} = 0$. Accordingly,define the out-degree of node $i$ as $d_i^o=\sum_{j=1}^Na_{ji}$ and the out-degree matrix as $D^o = {\rm diag}\{d_i^o\}\in {\bf R}^{N\times N}$. Then,the graph column Laplacian matrix can be defined as $L^o = D^o-\mathcal{A}^{\rm T}$.

A node is balanced if its in-degree equals its out-degree. A directed graph is balanced if all its nodes are balanced. Since for undirected graph,$\mathcal{A}^{\rm T} = \mathcal{A}$,all undirected graphs are balanced.

In a directed graph,a sequence of successive edges in the form $\{(v_i,v_k),(v_k,v_l),\cdots,(v_m,v_j)\}$ is a direct path from node $i$ to node $j$. An undirected path is defined similarly. A digraph is said to have a spanning tree,if there is a root node $i_r$,such that there is a directed path from the root to any other node in the graph. A digraph is said to be strongly connected,if there is a direct path from node $i$ to node $j$,for all distinct nodes $v_i$,$v_j \in \mathcal{V}$. A digraph has a spanning tree if it is strongly connected,but not vice versa. A digraph (or undirected graph) is said to be connected,if for any orderless pair of nodes,there is a directed (or undirected) path connecting them. For a digraph,its underlying graph is the graph obtained by replacing all directed edges with undirected edges. A digraph is weakly connected if its underlying graph is connected. More details can be found in the book Graph Theoretic Methods in Multiagent Networks^[33].

III. COORDINATION OF HIGH-ORDER LINEAR SYSTEMS A.Coordination of High-order Integrator Linear Systems

Basic high-order linear multi-agent systems can be represented by integrator chained systems in strict-feedback form as

\begin{align} \dot{\xi}_{i}^{(0)} &=\xi_{i}^{(1)},\nonumber\\ &\quad \!\!\!\!\vdots \\ \dot{\xi}_{i}^{(l-2)} &= \xi_{i}^{(l-1)},\notag \end{align} \begin{align} \dot{\xi}_{i}^{(l-1)}=u_i, \nonumber \end{align}

(2)

where $\xi_i^{(k)}\in {\bf R}^m$,$k=0,1,\cdots,l-1$,$l\geq 3$ denotes the states of the systems,$u_i \in {\bf R}^m$ is the control input of the $j$-th system,and $\xi_i^{(k)}$ denotes the $k$-th derivative of $\xi_i$ with $\xi_i^{(0)}$ $=$ $\xi_i$. This model generalizes the single-integrator and double-integrator dynamics.

The basic idea of a consensus algorithm is to impose similar dynamics on the information states of each high-order system. If the communication network among systems allows continuous communication or if the communication bandwidth is sufficiently large,then the information state update of each system is modeled using a differential equation. On the other hand,if the communication data arrive in discrete packets,then the information state update is modeled using a difference equation. The fundamental consensus algorithms for high-order linear systems can be shown as follows

\begin{align} % 3 u_i=-\sum\limits_{j=1}^ng_{ij}k_{ij}\left[\sum\limits_{k=0}^{l-1}\gamma_k\left(\xi_i^{k}-\xi_j^{k}\right)\right],\quad i\in \{ 1,\cdots,n \}, \end{align}

(3)

where $k_{ij} >0$,$\gamma_k >0$,$g_{ii} = 0$,and $g_{ij}$ is $1$ if information flows from system $j$ to system $i$ and $0$ otherwise. Consensus is reached among the $m$ systems if $\xi_i^{(k)} \rightarrow \xi_j^{(k)}$,$k= 0,1,\cdots$,$l-1$, $\forall i\neq j$. Let $\xi=[\xi_1,\cdots,\xi_n]^{\rm T}$. By applying consensus algorithm (3),(2) can be written in matrix form as

\[\left[ {\begin{array}{*{20}{c}} {{{\dot \xi }^{(0)}}} \\ {{{\dot \xi }^{(1)}}} \\ \vdots \\ {{{\dot \xi }^{(l - 1)}}} \end{array}} \right] = \left( {\Gamma \otimes {I_m}} \right)\left[ {\begin{array}{*{20}{c}} {{\xi ^{(0)}}} \\ {{\xi ^{(1)}}} \\ \vdots \\ {{\xi ^{(l - 1)}}} \end{array}} \right].\]

(4)

Due to the characteristics of multi-agent systems,the closed-loop system can be expressed by the vector form,therefore matrix theory has been frequently used in the stability analysis of linear systems$'$ distributed coordination. As a fundamental cooperative control problem,the leaderless consensus was considered first in literature. In ^{[34, 37]},Ren et al. proposed a class of $l$-order $(l\geq3)$ consensus algorithms and showed necessary and sufficient conditions under which each information variable and their high-order derivatives converged to common values. They also introduced the idea of high-order consensus with a leader and the concept of an $l$-order model-reference consensus problem,where each information variable and their high-order derivatives not only reach consensus but also converge to the solution of a prescribed dynamic model. However,it cannot solve the rendezvous problem because of the non-zero velocity when consensus is achieved. Jiang et al. investigated the consensus problem for multi-agent systems with individual agents modeled by high-order integrators under fixed/switching topology and zero/nonzero communication time-delays^[38]. Consensus of high-order integrators multi-agent systems with time-delays and switching topologies were studied by Yang et al. It was shown that consensus can be reached with arbitrarily bounded time-delays even though the communication topology might not have spanning trees^[39]. Zhang et al. also investigated consensus of high-order multi-agent systems. Different from the existing protocols in ^{[34, 37]},a dynamic neighbour-based protocol was proposed,which only uses the relative information of the first states of agents by extending second-order protocols from the literature. Sufficient conditions were derived under which all agents reach consensus asymptotically ^[40]. He et al. proposed a linear consensus protocol for solving such a consensus problem, which includes two parts: a feedback controller and interactions from the neighbours. A sufficient and necessary condition for consensus in high-order systems was obtained. They proposed the control protocol $u_i=u_{i1}+u_{i2}$,where $u_{i1}=b\sum\nolimits_{k=1}^{l-1}\xi_i^k$ is the feedback controller,$b$ is a non-zero constant to be determined. And $u_{i2}=\sum\nolimits_{j=1}^{n}c_{ij}\big (\sum\nolimits_{k=1}^{l-2}\gamma_k(\xi_j^k-\xi_i^k)\big )$. As special cases,the criteria for second- and third-order systems were given,in which the exact relationship between feedback gain and system parameters was established. The introduced parameters in the consensus protocol makes the design of the feedback gain more flexible^[41]. Leader-following consensus problems of high-order multi-agent linear systems with noises and time delays have been discussed in ^[42]. Under the condition that the topologies are undirected,some consensus conditions were obtained.

The literature mentioned above are under the matrix theory approaches based framework,for the model of the multi-agent systems are high-order linear systems in a chained form. Moreover, there are still some other methods to deal with the high-order linear system cooperative control problems,such as the famous backstepping technique. In the backstepping technique combining graphic theory,owing to the structure character of the lower-triangular strict-feedback formed system,the high-rank state of each differential equation is used as virtual control. Then,the consensus control problem for the higher-order multiple systems can be broken into a sequence of design problems for multiple lower order subsystems. In addition,the backstepping technique combining adaptive control method,sliding mode control,neural network method,etc. can solve not only linear systems control problems but also nonlinear control problems^{[36, 43]}.

B.Coordination of High-order Linear Systems with Disturbance

In some practical applications,multi-agent systems are often subject to various disturbances such as actuator bias, measurement/calculation errors,and variation of communication topology.

A class of high-order linear system with external disturbances can be described as follows^{[44, 45, 46, 47, 48]}

\begin{align} \dot{\xi}_{i}^{(0)} &= \xi_{i}^{(1)},\nonumber\\ &\vdots \nonumber\\ \dot{\xi}_{i}^{(l-2)} &= \xi_{i}^{(l-1)},\nonumber\\ \dot{\xi}_{i}^{(l-1)} &= u_i+w_i, \end{align}

(5)

where $i$ $(1 \leq i \leq n)$ denotes the index number of the system,$\xi_{i}$ $=$ $[\xi_{i}^{(0)},\xi_{i}^{(1)},\cdots,\xi_{i}^{(l-1)}]^{\rm T}\in {\bf R}^l$ denotes the states of the $i$-th system. $u_i \in {\bf R}$ is the control input of the $i$-th system,and $w_i$ is the external disturbance.

As for the multi-agent systems with external disturbance,Mo et al. provided the convergence analysis of the consensus problem for multi-agent systems with time delays and external disturbance^[44]. They proposed a dynamic neighbour-based protocol as follows

\begin{align} u_i(t) = -\sum\limits_{k=0}^{l-1}g_k\sum\limits_{j \in N_i(t)}a_{ij}\left(\xi_i^{(k)}(t-\tau)-\xi_j^{(k)}(t-\tau)\right), \end{align}

(6)

for any $i \in \{1,2,\cdots,n\}$,where $a_{ij}>0$ denotes the edge weight,$g_k>0$ for $k=0,1,\cdots,l-1$,and $\tau>0$ denotes the communication delay. The output functions were defined to combine the relative information by computing from an average of the relative displacements of all agents as follows

\begin{align} z_{il}(t) =&\ \frac{1}{n}\sum\limits_{j=1}^n\left[\xi_i^{(l-1)}(t)-\xi_j^{(l-1)}(t)\right] =\nonumber \end{align}\begin{align} &\ \xi_i^{(l-1)}(t)-\frac{1}{n}\sum\limits_{j=1}^n\xi_j^{(l-1)}(t). \end{align}

(7)

And the $H_{\infty}$ performance index was defined as follows

\begin{align} J = \int_0^{\infty} \left[z^{\rm T}(t)z(t)-\gamma^2w^{\rm T}(t)w(t)\right]{\rm d}t, \end{align}

(8)

where $\gamma$ is a given positive constant. The approach used in ^[44] does not need any model transformation,which is different from the results of first-order^[45] and second-order multi-agent systems^[46].

Liu et al.^[47] studied the output consensus problem of the $l$th-order multi-agent systems with external disturbances in networks with fixed and switching directed topologies,respectively. A controlled output was defined to measure the disagreement of each agent$'$s measured output to the average one of all agents. The controlled output functions were similar to (7) in ^[44]. The consensus problem can be transformed into an $H_\infty$ control problem. Then,a distributed protocol was proposed for each agent using its own information and its neighbors$'$ measured outputs,and a closed-loop system was derived with a singular state matrix,which resulted in invalidation of the traditional $H_{\infty}$ theory. Then,they conducted a model transformation by two steps,and obtained an equivalent nonsingular reduced-order system regarding the $H_{\infty}$ performance,based on which consensus of the multi-agent system was investigated. Specifically,for directed networks with fixed topology,a necessary and sufficient condition was obtained to ensure the output consensus with a given $H_\infty$ performance index. This consensus controller is different from the literature mentioned above either; for agent $i$,only its neighbors$'$ measured outputs $y_j(t)$ $=$ $\xi_j^{(1)}$ $(j\in \mathcal{N}_i)$ are directly obtainable,and the other variables of neighbors would have to be estimated. Thus,it can use the available information $\xi_j^{(1)}$ $(j\in \mathcal{N}_i)$,instead of all-order derivatives of $\xi_j^{(1)}(t)$,to design

\begin{align} u_i(t)=\sum\limits_{k=2}^l-K_k\xi_i^{(k)}(t)+K_1\sum\limits_{j\in \mathcal{N}_i}a_{ij}(\xi_j^{(1)}-\xi_i^{(1)}(t)), \end{align} for $i=1,2,\cdots,n$,where $K_k \in {\bf R}$ $(k=1,2,\cdots,l)$ are consensus gains. The role of local negative feedback $\sum\nolimits_{k=2}^l$ $=$ $-K_k\xi_i^{(k)}(t)$ is to decrease the modulus of high-order variables $\xi_i^{(k)}(t)$,$k=2,3,\cdots,l$.

(9)

However,how to deal with stochastic disturbances in high-order multi-agent systems,or even with stochastic communication topologies,is still a challenging problem.

C.Coordination of Discrete-time High-order Linear Systems

Many practical systems are with discrete-time models,and the high-order discrete-time multiple ones can be described as follows

\begin{align} \xi_i^{(0)}(k+1)& = \xi_i^{(0)}(k)+\xi_i^{(1)}(k)T,\nonumber \\ &\vdots & \\ \xi_i^{(l-2)}(k+1)& =\xi_i^{(l-2)}(k)+\xi_i^{(l-1)}(k)T, \nonumber \\ \xi_i^{(l-1)}(k+1)& =\xi_i^{(l-1)}(k)+u_i(k)T,\nonumber \end{align}

(10)

Suppose that the multi-agent systems (10) considered consists of $N$ discrete-time agents. Each agent at instant $kT$ is regarded as a node in a directed graph $ \mathcal{G}(kT) =(\mathcal{V}$, $\mathcal{E}(kT),\mathcal{A}(kT))$,where $T>0$ is the sampling time,$\mathcal{V}=$ $\{v_1$,$\cdots$,$v_N \}$ is the set of nodes,$\mathcal{E}(kT)\subseteq \mathcal{V} \times \mathcal{V}$ is the set of edges,and $\mathcal{A}(kT) = [a_{ij}(kT)]$ is a weighted adjacency matrix. Each edge $(v_j,v_i)\in \mathcal{E}(kT)$ denotes an information link from agent $v_i$ to agent $v_j$ at instant $kT$. The set of the neighbours of the $i$th agent and the Laplacian of the graph $\mathcal{G}(kT)$ at time $kT$ are represented by $N_i(kT)$ and $L(kT)$,respectively.

Different from ^[14],the rule for the discrete-time multi-agent high-order systems is given as

\begin{align} u_i(k)=&-\sum\limits_{j=1}^{l-1}p_j\xi_i^{(j)}(k) -\nonumber\\ & \sum\limits_{s_j \in N_i(k)}a_{ij}(k)(\xi_i^{(0)}(k)\!-\!\xi_j^{(0)}(k-\tau_{ij})), \end{align}

(11)

for any $i \in \{1,2,\cdots,n\}$,where $p_j>0$ for $j=1,2,\cdots,l-1$,$a_{ij}$ $>$ $0$ denotes the edge weight chosen from a finite set $\bar{\alpha}$ and $\tau_{ij}(t) \in {\bf Z}_{+}$, $\tau_{ij}(t)\leq \tau_{\rm max}$ $(i\neq j)$ is the communication time-delay from $v_j$ to $v_i$ for a given positive constant $\tau_{\rm max}$. Sufficient conditions were derived for the introduced rule to make all agents reach consensus under dynamically changing topologies in ^[49]. It is shown that consensus can be reached,even when the communication topologies are dynamically changing and the corresponding communication graphs have no spanning trees. It is also shown that the communication time-delays do not affect the stability of the multi-agent systems. In ^[50], leader-following consensus of discrete-time multi-agent systems was considered. And an observer-based protocols was proposed.

D.Coordination of General High-order LTI Systems

Recently,a class of general high-order linear time-invariant (LTI) systems has been considered^{[51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65]}. In this kind of multi-agent systems,all the agents have identical multi-input multi-output (MIMO) linear dynamics which can be of any order. The dynamics of the $i$th agent is described by

\begin{align} &\dot{x}_i=Ax_i+Bu_i,\nonumber \\ &y_i=Cx_i,\ i=1,\cdots,N, \end{align}

(12)

where $x_i\in {\bf R}^n$ is the state,$u_i\in {\bf R}^p$ the control input,and $y_i\in {\bf R}^q$ the measured output. $A$,$B$, $C$ are constant matrices with compatible dimensions,where $C$ is assumed to have full row rank.

In Sections III-A - III-C,the protocols are based upon all the states of neighboring agents. However,in terms of the general high-order LTI systems,the agent should have inputs as many as the order of the system^[51]. The output feedback consensus problem has been considered by Scardovi and Sepulchre^[52],where all the states of the observer for each agent need to be transmitted to the neighbors,so that the quantity of the transmitted information is the same as the state feedback case. The work by Tuna^[53] discussed various conditions for achieving consensus by output feedback,but was limited to static output feedback cases. On the basis of the above work,Seo et al.^[54] studied the consensus problem in a generalized environment in the sense that each agent is a multi-input multi-output linear dynamic system which is stabilizable and detectable,and that a dynamic consensus algorithm is proposed that uses only the output information (rather than the full state) from the neighboring agents. The output feedback consensus problem for $N$ identical linear dynamics was solved in a unified manner. By the way,the authors first showed that consensus problem is solved if there exists a stable dynamic filter which simultaneously stabilizes $N-1$ linear dynamic systems in a special form. Next,they proved that there exists such a filter under a very general condition and showed how to construct the filter. In ^[53], it was assumed that agent $i$ collected the output information of its neighboring agents by the following rule

\begin{align} z_i(t)=\sum\limits_{j\in \mathcal{N}_i}\alpha_{ij}(y_j(t)-y_i(t))=-\sum\limits_{j \in \mathcal{N}}l_{ij}y_j(t), \end{align}

(13)

in which $\mathcal{N}_i=\{j \in \mathcal{N}:\alpha_{ij}\neq 0\}$. They also assumed that the collected information $z_i(t)$ is filtered by a stable filter $\kappa (s)$,and is fed back to agent $i$ by

\begin{align} u_i=\kappa(s)z_i=\kappa(s)\sum\limits_{j\in \mathcal{N}_i}\alpha_{ij}(y_j-y_i)=-\kappa(s)\sum\limits_{j\in \mathcal{N}}l_{ij}y_j. \end{align}

(14)

Li^[55] considered the consensus problems for both continuous- and discrete-time linear multi-agent systems with directed communication topologies. Distributed reduced-order observer-based consensus protocols were proposed,based on the relative outputs of neighboring agents. A multi-step algorithm was presented to construct a reduced-order protocol,under which a continuous-time multi-agent system whose communication topology contains a directed spanning tree can reach consensus. The algorithm in ^[55] was further modified to achieve consensus with a prescribed convergence rate. These two algorithms have a favorable decoupling property. In light of the modified algebraic Riccati equation,an algorithm was then given to construct a reduced-order protocol for the discrete-time case. In ^[56],Wieland et al. considered the problem of reaching static or dynamic consensus over fixed interconnection topologies for multi-agent systems with agents modelled as general LTI systems. The consensus condition of the whole multi-agent systems was derived as stability of some matrices which consist of the agent dynamics and the spectrum of the Laplacian of the corresponding graph. In addition,a meaningful interpretation of the role of the adjoint null space of the Laplacian matrix was given. As far as design was concerned, systematic methods were proposed to choose the gains in the consensus algorithm such that the multi-agent systems reached consensus asymptotically with prespecified convergence rate. In terms of consensus seeking problem of high-order LTI multi-agent systems with fixed and switching topologies,Jiang et al.^[57] gave some results. A necessary and sufficient condition was derived for the consensus of the multi-agent systems with fixed topology and zero time-delay. The protocol designs were also discussed respectively for directed and undirected topologies. Under switching topology and zero time-delay,a sufficient condition for the consensus of the multi-agent system was established. In ^[58],Zeng et al. discussed the consensus problem of a group of general linear agents under an undirected topology. Both input and communication delays were taken into account. The factorization of the characteristic equation of the system into decreased-order factors relied only on the set of eigenvalues of a matrix that describes the structure of the network topology,and simplified the stability analysis considerably. Tang et al.^[59] investigated hierarchical distributed control for multi-agent systems using approximate simulation. They proposed the distributed control with two parts: one is the local control for coordination,while the other is from the virtual leader. In ^[60],consensus of high-order linear multi-agent systems using output error feedback was considered. Wang et al. investigated an observer based approach to design a dynamic output error feedback consensus control law.

Recently,one new direction is the formation swarm control for multiple LTI systems. Xi et al.^{[61, 62, 63, 64]} considered consensus analysis and design problems for high-order linear time-invariant swarm systems. In ^[61],consensus problems for high-order continuous-time linear time-invariant swarm systems with directed interaction topologies were investigated. A method to deal with consensus problems was proposed based on state space decomposition. Two subspaces of a complex space were introduced,namely a consensus subspace and a complement consensus subspace. Based on this decomposition,necessary and sufficient conditions for consensus and consensualizability were presented,respectively. An approach for determining and designing the consensus function which may be time-varying was shown. The applications of theoretical results to multi-agent supporting systems were also studied. In ^[62],the swarm stability problem of high-order linear time-invariant swarm systems with directed graph topology was dealt with. Consensus problems can be regarded as a specific type of swarm stability problem.

Necessary and sufficient conditions for both swarm stability and consensus were also presented. In ^[63],consensus analysis and design problems for high-order linear time-invariant swarm systems with time-varying delays were dealt with. First,a consensus subspace and a complement consensus subspace were introduced. By state projection onto the two subspaces,consensus problems were converted into simultaneous stabilization problems of multiple time-delayed subsystems with low dimensions,and a method to analyze and design the consensus function was given. Then,sufficient conditions for consensus and consensualization were presented,which included only four linear matrix inequality constraints. At present,it remains a challenging problem to deal with high-order swarm coordinative control problems.

IV. COORDINATION OF HIGH-ORDER NONLINEAR SYSTEMS

As for the high-order multi-agent systems,most of the existing reseaches are only concerned with the integrators chained systems with linear strict-feedback form. Whereas,almost all physical systems are inherently nonlinear,for example, wheeled mobile robots,dynamics of manipulators,etc.,thus the research of cooperative control for high-order nonlinear systems is more practical. What is more,in many practical applications, the dynamics of the systems are usually not only nonlinear but also with uncertainties,thus solving consensus problems for multiple high-order uncertain nonlinear systems would make great sense for practical applications. The matrix theory approaches mentioned above are not applicable to many scenarios,especially for nonlinear systems. Thus the cooperative control of high-order nonlinear systems with uncertainties is more challenging than that of the certain high-order linear ones. Today,there are still few results on cooperative control problems of high-order nonlinear systems.

A.Coordination of Typical High-order Nonlinear Systems

The typical high-order nonlinear multi-agent systems can be described by

\begin{align} &\dot{x}_{ij}=f_i(\bar{x}_{ij})+x_{i+1,j},\nonumber\\ &\dot{x}_{nj}=f_n(\bar{x}_{nj})+u_j, \end{align}

(15)

for $i=1,\cdots,n-1$,where $\bar{x}_{ij}=[x_{1j},\cdots,x_{ij}]^{\rm T}$. The function $f_i$ is smooth and is assumed to be known. $u_j \in {\bf R}$ is the control input of system $j$. In ^{[66, 67]},consensus and tracking problems of multiple high-order nonlinear systems with the model of (15) were considered. Distributed control laws were proposed with the aid of robust control theory,approximation-based control theory and backstepping technique as follows

\begin{align} u_j = \sum\limits_{i \in \mathcal{N}_j}a_{ji}(z_{nj}-z_{ni})-f_n(\bar{x}_{nj})+\dot{\alpha}_{nj}(\bar{x}_{n-1,j}), \end{align}

(16)

for $1 \leq j\leq m$,where $a_{ji}=a_{ij}\geq \delta >0$, $z_{*j}=x_{*j}-\alpha_{*j}$ are error variables of the $j$th system which are defined by backstepping technique,where $x_{*j}=[x_{1j},x_{2j},\cdots,x_{nj}]^{\rm T}$,and $\alpha_{*j}$ $=$ $[\alpha_{1j},\alpha_{2j},\cdots,\alpha_{nj}]^{\rm T}$ are the virtual controllers of the $j$-th system in each order, respectively. The protocol (16) makes the states of a group of systems converge to a stationary point. For tracking problem,a robust/adaptive control law (17) were proposed such that the states of the systems could converge to the state of the reference systems

\begin{align} u_j=&-\sum\limits_{i\in \mathcal{N}_j}a_{ji}(z_{nj}-z_{ni})-\mu_j b_j(z_{nj}-z_{n0})+\Phi_j(t)^{\rm T}\hat{\theta}_j -\nonumber\\ &\ f_n(\bar{x}_{nj})+\dot{\alpha}_{nj}(\bar{x}_{n-1,j}),\ \ 1\leq j \leq m, \end{align}

(17)

where constant $b_j \geq \delta >0$,$\mu_j=1$ if the states of the reference systems is available to system $j$,$\mu_j=0$ otherwise. $\Phi_j(t)^{\rm T}\hat{\theta}_j$ is the key item for consensus seeking by adaptive control theory.

Although the backstepping has become one of the most popular design methods for the strict-feedback systems,it has the "explosion of complexity" problem,which is caused by the repeated differentiations of virtual controllers in each backstepping design step. To overcome this problem,a dynamic surface design technique combing neural network technique was proposed for strict feedback systems by Yoo in ^[68],in which local distributed consensus controller were designed for the consensus tracking problem of model (15). The graph-based error surfaces for the distributed dynamic surface design were used to guarantee the consensus tracking between the followers and one leader. The function approximation technique using neural networks was employed to compensate the unknown nonlinearities unmatched in the control input of followers. The design procedure on the $i$th follower contains $n_i$ steps.

Compared to the model of (15),the following high-order uncertain nonlinear multi-agent systems are more general

\begin{align} &\dot{x}_{ij}= f_{ij}(\bar{x}_{ij})+x_{i+1,j}+\theta_j^{\rm T}\phi_{ij}(\bar{x}_{ij}),\nonumber\\ &\dot{x}_{nj}= f_{nj}(\bar{x}_{nj})+\theta_j^{\rm T}\phi_{nj}(\bar{x}_{nj})+u_j, \end{align}

(18)

where $\bar{x}_{ij}=[x_{1j},\cdots,x_{ij}]^{\rm T}$. The functions $f_{ij} \in {\bf R}$ and $\phi_{ij}$ $\in$ ${\bf R}^p$ are smooth and assumed to be known. $\theta_j \in {\bf R}^{p_j}$ is an unknown constant vector. $u_j \in {\bf R}$ is the control input of the $j$th system.

If $\theta_j=\theta$,distributed adaptive consensus algorithms can be proposed with the aid of backstepping technique for multiple $n$-order nonlinear systems in the strict feedback form. Dong^[69] considered the consensus problem of model (18) by using backstepping technique and adaptive control theory,in which the order of each system is only three. If the order of systems is larger than three,the proposed procedure does not work. For the reason that,as is well known,backstepping is a recursive design procedure whose complexity increases drastically with the order of the systems. Adding a power integrator design approach is an useful method to deal with some high-order lower-triangular system problems ^{[70, 71, 72, 73]}. However,the approach has very strict conditions for use. And the main existing results are limited in single system. At present,it remains a challenging problem to design an appropriate local controller for (18).

B.Coordination of High-order Nonlinear Systems in the Brunovsky Form

Another typical high-order nonlinear multi-agent systems are described as the Brunovsky form. Each agent is modeled by a higher-order integrator incorporating unknown nonlinear dynamics and an unknown disturbance as follows

\begin{align} &\dot{x}_{ij} = x_{i+1,j},\nonumber\\ &\dot{x}_{nj} = f_j(x_j)+u_j+\zeta_j, \end{align}

(19)

for $i=1,\cdots,n-1$,where $x_{ij} \in {\bf R}$ is the $i$th state of agent $j$; $x_j=[x_{1j},\cdots,x_{nj}]^{\rm T}$ is the state vector of agent $j$; $f_j(\cdot)$: ${\bf R}^n$ $\rightarrow$ ${\bf R}$ is locally Lipschitz in ${\bf R}^n$ with $f_j(0)=0$,and it is assumed to be unknown; $u_j \in {\bf R}$ is the control input; and $\zeta_j$ $\in$ ${\bf R}$ is an external disturbance,which is unknown but bounded.

Zhang et al.^[35] developed a practical design method for cooperative tracking control of (19) with a dynamic leader. The communication network is a weighted directed graph with a fixed topology. The leader node is modeled as a higher-order nonautonomous nonlinear system. It acts as a command generator giving commands only to a small portion of the networked group. A robust adaptive neural network controller was designed for each follower node such that all follower nodes ultimately synchronize to the leader node with bounded residual errors. Fewer assumptions were made in that paper and the requirement of graph topology is relaxed such that the augmented graph has a spanning tree. This means the original graph may be disconnected,as long as the leader node pins into the proper nodes in each disconnected component. This is a necessary condition and is less stringent than strong connectedness.

Most of the consensus algorithms available in the literature are applicable to asymptotic consensus. Asymptotic consensus implies that the differences between agents states converge to zero as time goes to infinity. In many time critical applications,it is required that consensus should be reached in a finite time. Khoo et al.^{[74, 75]} introduced a set of sliding variables for multi-agent networks with both directed and undirected network topologies as

\begin{align} s_i^h = (e_i^h)^{\frac{1}{Q_i}}-{x_{id}^h}^{\frac{1}{Q_i}}, \end{align}

(20)

where $Q_i=p_i/q_i$,with $p_i$ and $q_i$ being odd positive integers satisfying $1 > p_1/q_1 > \cdots >p_n/q_n > 0$. $x_{id}^d$ represents the virtual control law of agent $h$ with $x_{1d}^h=0$ and

\begin{align} e_i^h = \sum\limits_{j=1,j\neq h}^m a_{hj}(x_i^h-x_i^j)+b_h(x_i^h-x_i^0), \end{align}

(21)

where $b_h$ denotes the connection weight between agent $h$ and the leader (agent $0$). If agent $h$ is connected to the leader, then $b_h>0$ and $b_h=0$ otherwise. And high-order leader-follower consensus can be guaranteed on these sliding surfaces. The fast finite-time control Lyapunov functions for control design were employed in ^{[74, 75]} so that faster finite time convergence of the sliding variables could be obtained,and the design of nonsmooth control laws ensured the fast finite-time reachability of this set of sliding surfaces,which also implies fast finite-time consensus of multi-agent network. This method combines backstepping technique,sliding mode control and adding a power integrator approach.

C.Coordination of High-order Nonholonomic Systems

A nonholonomic system in physics and mathematics is a system whose state depends on the path taken to achieve it. Such a system is described by a set of parameters subject to differential constraints,such that when the system evolves along a path in its parameter space (the parameters varying continuously in values) but finally returns to the original set of values at the start of the path,the system itself may not have returned to its original state.

Many mechanical devices are described by systems with underactuation. Therefore,it is important to develop appropriate consensus algorithms for underactuated systems. Nonholonomic mobile robots are typical underactuated systems which have been studied for many years^{[76, 77, 78, 79, 80, 81]}. However,most of the existing literature only focus on the basic and simple nonholonomic wheeled mobile robots which move on a horizontal planes; the motion of these systems is described by

\begin{align} \dot{x}_j=v_j\cos{\theta_j},\ \ \dot{y}_j=v_j\sin{\theta_j},\ \ \dot{\theta}_j=w_j, \end{align}

(22)

where $(x_j,y_j)$ is the position of robot $j$ in a coordinate system,$\theta_j$ is the orientation of robot $j$,$v_j$ is the speed of robot $j$,and $w_j$ is the angular velocity of robot $j$. The control signals are $v_j$ and $w_j$.

Consider a class of high-order nonholonomic systems in the following power chained form:

\begin{align} &\dot{x}_{0j} = u_{0j}^{p_0},\nonumber \\ &\dot{x}_{1j} = x_{2j}^{p_1}u_{0j}^{q_1} ,\nonumber \\ &\qquad \vdots \\ &\dot{x}_{n-1,j} = x_{nj}^{p_{n-1}}u_{0j}^{q_{n-1}},\nonumber \\ &\dot{x}_{nj} = u_{1j}^{p_n},\nonumber \end{align}

(23)

where $x_j = (x_{0j},x_{1j},\cdots,x_{nj})^{\rm T} \in {\bf R}^n $ and $u_j =(u_{0j},u_{1j})^{\rm T}$ $\in$ ${\bf R}^2$ represent the $j$th system states and control input,respectively, $p_i$,$i = 0,\cdots,n$,and $q_k$,$k = 1,\cdots,n-1$,are positive integers.

The system (23) under consideration represents a class of inherent nonlinear systems and includes the well-studied chained-form nonholonomic system as its special case. The main difficulty of controlling this type of nonlinear systems lies in that the systems are neither stabilizable by any time-invariant continuous state feedback (as Brocketts necessary condition fails to be satisfied) nor affine in the control inputs,and each system only has its local information.

Lin^[82] addressed the problems of almost asymptotic stabilization and global asymptotic regulation for the single system of (23),by using adding a power integrator technique. The nonholonomic system considered in ^[82] is not necessarily affine in the control variables and therefore cannot be handled by existing methods. Sufficient conditions are presented under which a discontinuous state feedback control law (respectively,a switching controller) can be recursively constructed. Today,it is still an open question.

D.Remarks

In terms of the inherent nonlinear characteristics of many practical multi-agent systems,Liu et al.^[83] discussed consensus problems for multi-agent systems with inherent nonlinear dynamics under directed topologies. A variable transformation method is used to convert the consensus problem to a partial stability problem. Both first-order and second-order systems are investigated under fixed and switching topologies,respectively. It is assumed that the inherent nonlinear terms satisfy the Lipshitz condition. Sufficient conditions on the feedback gains are given based on a Lyapunov function method. For first-order systems under a fixed topology,the consensus is achieved if the feedback gain related to the agents$'$ positions is large enough.

If the nonlinear system satisfies the controllable and involutive conditions,then there exists the diffeomorphism. The nonlinear system can be transformed into the form of high-order integrator by feedback linearization. Therefore,the high order linear consensus algorithm can be used to achieve consensus of the nonlinear systems that is feedback linearized. By satisfying the conditions that is related with network structure and gains of the consensus algorithm,the consensus of nonlinear systems is achieved through feedback linearziation and the linear consensus algorithm. In terms of feedback linearization for consensus of nonlinear multi-agent systems,Lee^[84] considered the consensus of high-order nonlinear systems which can be described as

\begin{align} \dot{x}= Ax+B\gamma(x)[u-\alpha(x)], \end{align}

(24)

where $A$ is $l \times l$,$B$ is $l \times p$,the pair $(A,B)$ is controllable,functions $\alpha:{\bf R}^l\rightarrow {\bf R}^p$ and $\gamma:{\bf R}^l \rightarrow {\bf R}^{p\times p}$ are defined in domain $D \in {\bf R}^l$ that contains the origin,and the matrix $\gamma(x)$ is nonsingular for every $x\in D$. It can be linearized by nonlinearity cancellation. A nonlinear term $\alpha(x)$ can be cancelled by subtraction,the control $u$ and the nonlinearity $\alpha(x)$ must always appear together as a sum of $u + \alpha(x)$.

To cancel a nonlinear term $\gamma(x)$ by division,the control u and the nonlinearity $\gamma(x)$ must always appear as a product of $\gamma(x)u$. If matrix $\gamma(x)$ is nonsingular in the domain of interest,then it can be cancelled by $u = \beta(x)v$,where $\beta(x)=$ $\gamma^{-1}(x)$ is the inverse matrix $\gamma(x)$. Therefore,the nonlinear system (24) can be converted into (12) via the state feedback linearization. Then,the cooperative control results for high-order LTI systems can be used in high-order linear systems. Chen et al.^[85] considered optimal full state feedback tracking control of unknown nonlinear systems. To deal with the unknown nonlinearities in the system,small learning regions are assigned online along the system trajectory in a manner dictated by a Lyapunov based self-organization method. In each of these regions, a local affine approximation is developed. A state observer-based approach adapts to the approximator parameters. With the aid of this state observer,analytic optimal controllers are proposed by solving corresponding linear quadratic control problems in each learning region.

V. DISCUSSION

Most previous studies on multi-agent systems studied the first-order and second-order dynamics. However,in engineering,many systems cannot be modeled by simple dynamics but high-order ones. Thus, coordination of high-order multi-agent systems has been a new heated topic recently. In most instances,the extension of cooperative control to high-order dynamics is not straightforward from the existing results with the first-order and second-order dynamics. In this paper,we review the main research and progress in distributed high-order linear and nonlinear multi-agent coordination,including the main kinds of models,cooperative control problems and approaches for high-order multi-agent systems.

Firstly,as for the high-order multi-agent systems,most of the existing literature are only concerned with the integrators chained systems with linear strict-feedback form. Due to the inherent characteristics of multiple linear systems,matrix theory approaches are used frequently in stability analysis. And the other techniques for stability analysis include dissipativity theory,nonsmooth analysis,eigenvalues analysis,backstepping technique,reduced-order observer control and especially Lyapunove functions etc. What is more,in many practical applications,the dynamics of the systems are usually not only nonlinear but also with uncertainties,thus solving consensus problems for multiple high-order uncertain nonlinear systems would make great sense for practical applications. Matrix theory approaches mentioned above are not applicable to many scenarios,especially for nonlinear systems. As the complexity of the nonlinear systems,it can not use a unified approach to deal with all the control problems. Thus,the cooperative control of high-order nonlinear systems with uncertainties is more challenging than that of the certain high-order linear ones.

Although the existing theoretical research and experiments have solved a number of technical problems in distributed high-order multi-agent coordination,there are still important and challenging research problems deserving further investigation. So far,the high-order multi-agent coordination and control are more a theoretical study,which needs to consider more physical truths. For example,the actual mechanical systems often have symmetry and nonholonomic constraints,etc.,in terms of satellite or spacecraft formation,it is necessary to consider its orbit constraints and drift. The high-order multi-agent theoretical results combining practical engineering approach also requires in-depth study and exploration. How to closely integrate the high-order multi-agent theory and engineering practice is still a challenging direction.