Variational AutoEncoder Model

Note

We will refer to Auto Encoders as AE.

Note

We will refer to Variational Auto Encoders as VAE.

What do you mean by variational?

VAEs are normal AEs with a twist: the encoded vector is constrained to be a noisy Gaussian distribution. In VAEs, during encoding of images, the encoded latents are assumed to be the mean, mean, and standard deviation, stddev, of some distribution corresponding to the image, and noises are added to the mean latent, with scale stddev to ensure that the model is robust.

Are VAEs better?

For generators, definitely. VAEs usually generates better than normal AEs do, because it has to learn to generate images when the latent is a little bit off (because of how the noises are added to mean).

However, for training a good compression model, VAEs usually cannot reduce the latent size as aggressively as AEs because of the same reason, it has to be more robust so more information has to be passed through to ensure that.