[lcc-0010]

In Robust control for dynamical systems with non-gaussian noise via formal abstractions, the following scheme for controlling dynamical systems is described:

1. Take a parametrized discrete-time dynamical system with noise - that is, something of the form x_{n+1} = F(x_n, u_n) + v_n, where the intended semantics is that u_n is the input chosen at each step, and v_n is randomly drawn from some distribution, and this whole thing lives in \mathbb {R}^n. For most of the paper the simple case where inputs live in \mathbb {R}^m and F is linear (not bilinear, but linear on \mathbb {R}^{n+m}) is cosidered, but this is not strictly necessary - what is necessary is the assumption that the noise term is independent of x_n and added on after computing the next term (that is - we could consider something like x_{n+1} = F(x_n,u_n,v_n) where v_n is random - that would require a more complicated method, I think).
2. Choose two things - a partition of \mathbb {R}^n into regions X_i and a "target point" in each d_i \in X_i - these regions will be taken to be convex and the target point should be central in a certain sense. The point here is that, if $I$ is the set of regions, this is a lens \binom {\mathbb {R}^n}{\mathbb {R}^n} \leftrightarrows \binom {I}{I} - the regions give you "high-level" information about where you are, and the points transform high-level control into low-level control.
3. Given this, for each region i \in I, say another region j is "reachable from i" if, for all x \in X_i, there exists a choice of u so that F(x,u) = d_j. Call this u(x,j)
4. Now the idea is, if we're at point x \in X_i, and we choose input u(x,j) for a reachable j, our next state is equal to d_j + v_n, regardless of what x is. This only works because the noise term is independent of the action at each step.
5. This means we can define a MDP where the states are I, the actions in state i are those a \in I reachable from i, and the transition probability P(i,a,j) (for i,j \in I, a \in I and a reachable from i) is the probability that you land somewhere in X_j when choosing u(x,a), given x \in X_i. u and the map which carries each x to the label of its equivalence class then constitute a lens which relates these two stochastic systems, meaning we've reduced the control problem to controlling this MDP

(The paper contains a further refinement, where you assume the distribution of u is not known, but has to be learned from a set of samples, and provides a method for robust control in this context).