Modeling, Simulation, and Optimization of Real Processes

## Non-Linear Observer Design - a data-driven optimal control perspective

#### Duration

01.11.2022 - 31.10.2025

#### Funding

DFG-Project 504768428

### Basic idea and applications

In order to apply our mathematical results to real life problems we first have to bridge the gap between theory and reality. This is done using mathematical models of so-called dynamical systems. In one of our examples such a dynamical system is given as a moving flock of birds. When observing a flock of birds in real life one notices that the paths of the individual animals depend on each other. They avoid collisions and also stay within a certain range of each other while flying in the same direction with similar speed. This behavior can be described using a mathematical model. More precisely, the positions and velocities of the birds are represented by a function that is obtained as the solution to a specific differential equation. This model enables us to analyze and predict the flock's movement using a computer. However, this approach has one fundamental flaw: No model describes the real system exactly. One major issue is that the models themselves are inaccurate. It is practically impossible to describe the complex behavior of the animals perfectly using mathematical formulas. Furthermore, real systems are prone to disturbances, in our example the trajectory of the birds will be influenced e.g. by wind.

We can improve the accuracy by augmenting the mathematical model with measured data obtained from the real life system. In our example one can look at the birds and measure their positions. The velocities however can not be measured directly but are only derived from the positions over time. The purpose of an observer is to combine the model and the measured data to allow a more accurate description of the flock's behavior.

There already exists a practically feasible solution to this issues that is applicable to a certain class of systems, namely linear systems. It was developed in the sixties and was for example applied in the Apollo mission of NASA. Until today it is the standard in the applied sciences to observe linear systems. Our goal is to generalize this approach to more complex, non-linear systems. The idea we are pursuing was also first proposed in the sixties, but due to lack of computing power and appropriate algorithms it was not discussed any further at the time.

Small flock of birds whose movement we aim to simulate using mathematical models

### Mathematical formulation and methodology

The so-called Mortensen observer is based on an optimal control problem and minimizes the energy of the disturbances in the system. It was proposed among others by R. E. Mortensen in [8]. The foundation of its design is the optimal control problem shown in (1). The quadratic cost functional J consists of the three inaccuracies of the system: The disturbances in the initial condition, in the dynamics (denoted by v), and in the measured data. It is minimized over all possible system disturbances and corresponding state trajectories x. It should be noted that the state equation evolves backwards in time.

The associated value function is defined in (2) and is of central importance to the theory and the numerical realization of the observer. This becomes apparent when looking at the definition of the observer trajectory in (3). Here the approximation of the real world system is defined as the pointwise minimizer of the energy of the disturbances in the system. It is shown to formally satisfy the observer equation (4).

The concept of the Mortensen observer has been investigated more recently and there are results available in the literature that apply to certain classes of systems. For example W. H. Fleming discussed the optimal control problem and associated value function in detail assuming that the right hand side of the system has globally bounded derivatives, see [12]. For systems with globally Lipschitz right hand side A. J. Krener presented convergence results in [13]. In [14] P. Moireau investigated a time discretized version of the observer, assuming that the right hand side is affine. A numerical realization of the Mortensen observer based on an approximation of the value function obtained using a neural net has been presented in [11].

Our goal is to drop the assumptions made on the system and instead only make local assertions. This means that we can consider more general non-linear systems but have to make assumptions on the accuracy of the model.

### Obtained results and current challenges

In our project we consider the theoretical background as well as the numerical realization of the Mortensen observer. This leads to two main questions we want to consider:

1. Under what assumptions is the expression in (3) well defined and satisfies equation (4)?

To answer this question it is crucial to consider the properties of the value function (2). We deploy well-known tools from sensitivity analysis such as the implicit function theorem to show local smoothness in the spatial variable. Using the Hamiltion Jacobi Bellman equation we further obtain local smoothness of the value function in time. By derivation of the HJB we can characterize the Hessian matrix of the value function evaluated along the model as the solution of a differential Riccati equation which shows positive definiteness. Due to the smoothness of the value function this property holds locally around the model and hence the value function is locally strictly convex.

Based on these observations we arrive at the main result of the first article that resulted from our project [1]: For sufficiently accurate models of the real world system the observer trajectory (3) is well defined and satisfies the observer equation (4).

2. How can the observer be realized numerically?

First note that via (3) and (4) two different characterizations of the observer are available. However, for both these approaches an appropriate approximation of the value function is essential to the realization. While in theory the value function is available as a solution of the HJB equation, it often is not practically feasible to numerically solve this equation for systems of medium and high state space dimension. Therefore we pursue the approach of evaluating the value function in chosen sample points which is done by solving the optimal control problem using a gradient descent method. The generated data is used to obtain a polynomial approximation of the value function using linear regression. This approximation can then be minimized according to (3) or it can be used to solve the observer equation (4).

The video shows a preliminary numerical result for the Cucker-Smale model which simulates  the movement of flocks of birds. The lines represent the trajectories of ten birds moving away from each other. Here our approximation of the Mortensen observer is shown in magenta.

### Related literature

Our results

[1] T. Breiten and J. Schröder, Local well-posedness of the Mortensen observer, 10.48550/arXiv.2305.09382
[16] T. Breiten and K. Kunisch and J. Schröder, Numerical realization of the Mortensen observer via a Hessian-augmented polynomial approximation of the value function, 10.48550/arXiv.2310.20321

Regularity of value functions

[2] P. Cannarsa and H. Frankowska, Value function and optimality conditions for semilinear control problems, Appl. Math. Optim., 26 (1992)
[3] P. Cannarsa and H. Frankowska, Some characterizations of optimal trajectories in control theory, SIAM J. Control. Optim., 29 (1991)
[4] T. Breiten and K. Kunisch and L. Pfeiffer, Feedback stabilization of the two-dimensional Navier-Stokes equations by value function approximation, Appl. Math. Optim., 80 (2019)
[5] T. Breiten and K. Kunisch and L. Pfeiffer, Taylor expansions of the value function associated with a bilinear optimal control problem, Ann. Inst. Henri Poincaré (C) Anal. Non Linéaire, 36 (2019)
[6] K. Malanowski, Application of the classical implicit function theorem in sensitivity analysis of parametric optimal control, Control Cybern., 27 (1988)
[7] K. Malanowski and H. Maurer, Sensitivity analysis of optimal control problems subject to higher order state constraints, Ann. Oper. Res., 101 (2001)

Observers

[8] R. E. Mortensen, Maximum-likelihood recursive nonlinear filtering, J. Optim. Theory Appl., 2 (1968)
[9] R. E. Kalman, A new approach to linear filtering and and prediciton problems, Trans. ASME Ser. D. J. Basic Engrg., 82 (1960)
[10] R. E. Kalman and R. S. Bucy, New results in linear filtering and prediction theory, Trans. ASME Ser. D. J. Basic Engrg., 83 (1961)
[11] T. Breiten and K. Kunisch, Neural network based nonlinear observers, Syst. Control Lett., 148 (2021)
[12] W. H. Fleming, Deterministic nonlinear filtering, Ann. Sc. norm. super. Pisa  - Cl. sci., 25 (1997)
[13] A. J. Krener, The convergence of the mimimum energy estimator, in "New trends in Nonlinear Dynamics and Control and their Applications", (2003)
[14] P. Moireau, A discrete-time optimal filtering approach for non-linear systems as a stable discretization of the Mortensen observer, ESAIM Control Optim. Calc. Var., 24 (2018)
[15] J. S. Baras and A. Bensoussan and M. R. James, Dynamic observers as asymptotic limits of recursive filters: special cases, SIAM J. Appl. Math., 48 (1988)