Separation principle in stochastic control

rdf:langString Separation principle in stochastic control

rdf:langString The separation principle is one of the fundamental principles of stochastic control theory, which states that the problems of optimal control and state estimation can be decoupled under certain conditions. In its most basic formulation it deals with a linear stochastic system with a state process , an output process and a control , where is a vector-valued Wiener process, is a zero-mean Gaussian random vector independent of , , and , , , , are matrix-valued functions which generally are taken to be continuous of bounded variation. Moreover, is nonsingular on some interval . The problem is to design an output feedback law which maps the observed process to the control input in a nonanticipatory manner so as to minimize the functional where denotes expected value, prime denotes transpose. and and are continuous matrix functions of bounded variation, is positive semi-definite and is positive definite for all . Under suitable conditions, which need to be properly stated, the optimal policy can be chosen in the form where is the linear least-squares estimate of the state vector obtained from the Kalman filter where is the gain of the optimal linear-quadratic regulator obtained by taking and deterministic, and where is the Kalman gain. There is also a non-Gaussian version of this problem (to be discussed below) where the Wiener process is replaced by a more general square-integrable martingale with possible jumps. In this case, the Kalman filter needs to be replaced by a nonlinear filter providing an estimate of the (strict sense) conditional mean where is the filtration generated by the output process; i.e., the family of increasing sigma fields representing the data as it is produced. In the early literature on the separation principle it was common to allow as admissible controls all processes that are adapted to the filtration . This is equivalent to allowing all non-anticipatory Borel functions as feedback laws, which raises the question of existence of a unique solution to the equations of the feedback loop. Moreover, one needs to exclude the possibility that a nonlinear controller extracts more information from the data than what is possible with a linear control law.