Example Generative Adversarial Network [efr-RUTY]

The idea of a generative adversarial network, or GAN (Reference [gan-paper]), is to generate synthetic data which appears to be drawn from the same distribution as a given dataset. The basic idea is to train two networks in parallel: a discriminator, which is trained to distinguish the generator's output from the real dataset, and a generator, which is trained to fool the discriminator. In each training step, one of two things happen:

1. We take a sample from the dataset, have the discriminator score it, and update it according to gradient descent to maximize the score.
2. We take a sample from the latent distribution, run the generator on it, then run the discriminator on the output. Then we update the discriminator with gradient descent to minimize the score, and update the generator with gradient descent to maximize the score in this case.

A GAN is naturally thought of as a sort of game played between two players, and hence has been an interesting example to study the parallels between the algebraic structure of machine learning and open games. However, since the game is essentially probabilistic, modeling this requires a category of lenses which contains both smooth manifolds, their tangent bundles and derivatives, as well as a good supply of stochastic maps. Additionally, the most natural way to model a GAN is as a morphism on a coproduct of spaces - hence we want coproducts in the lens category as well. The systems doctrine constructed in this section has all these features.

Fix a distribution \mu _D on \mathbb {R}^N, the data distribution. Typically this will be a discrete distribution given by a dataset, perhaps of images. Also fix a latent distribution \mu _L on \mathbb {R}^L. Typically this will be something like an uncorrelated Gaussian. Choose a generator network G: \mathbb {R}^{P_g} \times \mathbb {R}^L \to \mathbb {R}^N and a discriminator network D: \mathbb {R}^{P_d} \times \mathbb {R}^N \to [0,1]. In practice these are given by neural networks, but for now we require them simply to be smooth maps. (Actually, neural networks with ReLU activations are not smooth, but we ignore this subtlety for now)

Finally choose optimization algorithms T\mathbb {R}^{P_g} \leftrightarrows T^*\mathbb {R}^{P_g} and T\mathbb {R}^{P_d} \leftrightarrows T^*\mathbb {R}^{P_d} for the generator and discriminator networks. Given this data, we obtain a bisystem {T^*\mathbb {R}^N + T^*\mathbb {R}^L \choose \mathbb {R}^N + \mathbb {R}^L} \nrightarrow {* \choose \mathbb {R}^N \times [0,1]} with parameter space \mathbb {R}^{P_g} \times \mathbb {R}^{P_d} as follows:

1. First, there is a morphism \mathbb {R}^{P_g} \times (\mathbb {R}^N + \mathbb {R}^L) \to \mathbb {R}^N + \mathbb {R}^N given by (_, \iota _0(d)) \mapsto d, (p_g, \iota _1(l)) \mapsto G(p_g,l). Applying T^* to this, and composing with the optimization algorithm for the generator we obtain a bisystem T^*(\mathbb {R}^N + \mathbb {R}^L) \to T^*\mathbb {R}^N + T^*\mathbb {R}^N with parameter space \mathbb {R}^{P_g}. Call this \hat {G}
2. Second, there is an obvious morphism \mathbb {R}^{P_d} \times (\mathbb {R}^N + \mathbb {R}^N) \to ([0,1] + [0,1]) which simply runs the discriminator in both cases. Taking the reverse derivative of this, and applying the optimization algorithm for the generator, we obtain a bisystem T^*\mathbb {R}^N + T^*\mathbb {R}^N \to T^*[0,1] + T^*[0,1]. Call this \hat {D}
3. Now there is an endomorphism lens T^*\mathbb {R}^N \leftrightarrows T^*\mathbb {R}^N which is the identity 1_{\mathbb {R}^N} in the forwards direction, and multiplication by -1 in the backwards direction. Call this N.
4. The composite \hat {G} ; 1 + N ; \hat {N} is now a bisystem T^*(\mathbb {R}^N + \mathbb {R}^L) \to T^*[0,1] + T^*[0,1]. Finally, we compose this with the covector field [0,1] \to T^*[0,1] given by d \mathrm {id} in the first branch, and by - d \mathrm {id} in the second branch.

The semantics of this when given a data point x \in \mathbb {R}^N is to run the discriminator, then update according to the gradient of the output---that is, to maximize the output. Given a latent sample x \in \mathbb {R}^L, we first run the generator, then the discriminator. The discriminator is now updated in the opposite direction---to minimize the output on this value, while the generator is updated in the positive direction---to maximize the value of its generation on this sample.

So far we have worked inside the doctrine of smooth dynamical systems. However, passing into the stochastic doctrine just described, we may precompose these bisystems with the map I \to \mathbb {R}^N + \mathbb {R}^L given by sampling from \mu _D with probability p and from \mu _L with probability 1-p. The resulting dynamical system describes the training dynamics of the GAN.

(One can also consider more complicated versions of this, where the number p is controlled by another dynamical system. One can also embed a controller which weighs the gradients sent to the two networks, so that for example if the discriminator is too good, one slows down its learning rate compared to the generator for a while.)