Optimal Control and Estimation
Preface
1
The Optimal Control Formulation
1.1
The Basic Problem
1.2
Dynamic Programming and Principle of Optimality
1.3
Infinite-horizon Formulation
2
Exact Dynamic Programming
2.1
Linear Quadratic Regulator
2.1.1
Infinite-Horizon LQR
2.1.2
LQR with Constraints
2.2
Markov Decision Process
2.2.1
Bellman Optimality Equations
2.2.2
Value Iteration
2.2.3
Value Iteration with Barycentric Interpolation
3
Approximate Optimal Control
3.1
Fitted Value Iteration
3.1.1
Linear Features
3.1.2
Neural Network Features
3.1.3
Fitted Q-value Iteration
3.1.4
Deep Q Network
3.1.5
Deep + Shallow
3.2
Trajectory Optimization
3.2.1
Direct Single Shooting
3.2.2
Direct Multiple Shooting
3.2.3
Direct Collocation
3.2.4
Direct Orthogonal Collocation
3.2.5
Failure of Open-Loop Control
3.2.6
LQR Trajectory Tracking
3.3
Model Predictive Control
3.3.1
Turn Trajectory Optimization into Feedback Control
3.3.2
Controllability, Reachability, and Invariance
3.3.3
Basic Formulation for Linear Systems
3.3.4
Persistent Feasibility
3.3.5
Stability
3.3.6
Explicit MPC
3.4
Policy Gradient
4
Continuous-time Optimal Control
4.1
The Basic Problem
4.2
The Hamilton-Jacobi-Bellman Equation
4.3
Linear Quadratic Regulator
4.3.1
LQR Trajectory Tracking
4.3.2
LQR Trajectory Stabilization
4.4
The Pontryagin Minimum Principle
4.4.1
Numerical Solution of the TPBVP
4.5
Infinite-Horizon Problems
4.5.1
Infinite-Horizon LQR
4.6
Viscosity Solution
5
Stability Analysis
5.1
Autonomous Systems
5.1.1
Concepts of Stability
5.1.2
Stability by Linearization
5.1.3
Lyapunov Analysis
5.1.4
Invariant Set Theorem
5.1.5
Computing Lyapunov Certificates
5.2
Controlled Systems
5.3
Non-autonomous Systems
6
Output Feedback
6.1
Least-Squares Estimation
6.1.1
Linear Least-Squares Estimation
6.2
Kalman Filter
6.2.1
Steady-State Kalman Filter
6.2.2
Continuous-time Kalman Filter
6.3
Linear Quadratic Gaussian Control
6.3.1
Steady-state LQG
6.3.2
Continuous-time LQG
6.4
Nonlinear Filtering
6.4.1
Extended Kalman Filter
6.4.2
Unscented Kalman Filter
6.4.3
Particle Filter
6.4.4
Feedback Particle Filter
6.5
State Observer
6.5.1
General Design Strategy
6.5.2
Luenberger Template
6.5.3
State-affine Template
6.5.4
Kazantzis-Kravaris-Luenberger (KKL) Template
6.5.5
Triangular Template
6.5.6
Design with Convex Optimization
6.6
Observer Feedback
7
Geometric Vision
7.1
3D Rotations and Poses
7.1.1
Rotation matrices
7.1.2
Coordinate Frame
7.1.3
Representations of the rotations
7.1.4
Miscellaneous topics on rotations
7.2
The Pinhole Camera Model
7.3
Camera Pose Estimation
7.3.1
The P3P Problem
7.3.2
The PnP Problem
7.3.3
Global Optimality
7.3.4
Handling Outliers
7.4
Point Cloud Registration
8
Adaptive Control
8.1
Model-Reference Adaptive Control
8.1.1
First-Order Systems
8.1.2
High-Order Systems
8.1.3
Robotic Manipulator
8.2
Certainty-Equivalent Adaptive Control
9
Problem Sets
Acknowledgement
Appendix
A
Linear Algebra and Differential Equations
A.1
Linear Algebra
A.1.1
Matrix Exponential
A.1.2
Gradients
A.2
Solving an Ordinary Differential Equation
A.2.1
Separation of Variables
A.2.2
First-order Linear ODE
A.2.3
Gronwall Inequality
A.2.4
Matlab
B
Convex Analysis and Optimization
B.1
Theory
B.1.1
Sets
B.1.2
Convex function
B.1.3
Lagrange dual
B.1.4
KKT condition
B.2
Practice
B.2.1
CVX Introduction
B.2.2
Linear Programming (LP)
B.2.3
Quadratic Programming (QP)
B.2.4
Quadratically Constrained Quadratic Programming (QCQP)
B.2.5
Second-Order Cone Programming (SOCP)
B.2.6
Semidefinite Programming (SDP)
B.2.7
CVXPY Introduction and Examples
C
Linear System Theory
C.1
Stability
C.1.1
Continuous-Time Stability
C.1.2
Discrete-Time Stability
C.1.3
Lyapunov Analysis
C.2
Controllability and Observability
C.2.1
Cayley-Hamilton Theorem
C.2.2
Equivalent Statements for Controllability
C.2.3
Duality
C.2.4
Equivalent Statements for Observability
C.3
Stabilizability And Detectability
C.3.1
Equivalent Statements for Stabilizability
C.3.2
Equivalent Statements for Detectability
D
Algebraic Techniques and Sum-of-Squares
D.1
Algebra
D.1.1
Polynomials
D.1.2
Representation of nonnegative polynomial: Univariate case
E
The Kalman-Yakubovich Lemma
F
Feedback Linearization
G
Sliding Control
References
Published with bookdown
Optimal Control and Estimation
G
Sliding Control