Generative Models

Why generative models?

Humans can think of marvelous things. Look at Mozart’s ability to create music, and Van Gough’s prowess to paint! In machine learning words, we call these people good generators. They are able to generate stuff that are so good it’s called art.

Generative models strive to do the same for computers. It’s very different from the normal classification/regression problem (even though the models still use them under the hood), because in classification/regression, you are reducing the input (an image, a voice command) to something simpler. For classification, it’s a label. For regression, it’s a handful of numbers. While generative models want to create output that are complicated, and real.

How do generative models generate things?

A computer can simply generate things because it would look like garbage, a few real world data need to be used to tell the machine what are good real world examples.

In training the generator, we usually use several ways to correct the behavior of the model by providing incentive. For AE/VAE it’s the reconstruction loss. For GAN it’s the discriminator’s loss. You don’t need to know what it means yet, they will be introduced in the following sections, just have to bear in mind that even generative models require real world examples (contrary to common believes).