AutoEncoder Model

Why autoencoders?

Machine learning is all about information. AutoEncoder was initially developed to capture that information.

What do you mean by information? Well, information is the most crucial part of data. For example, if I’m trying to tell you: “You are a beautiful human being”. You’ll likely understand me when I say like a caveman: “You, beautiful, human!”. Notice that the second sentence is much shorter than the first one, yet it’s perfectly understandable by any person (and caveman). We say that the amount of information is the same in the first one and the second one. Weirdly enough, scientists like the second one (caveman version) better because it is shorter and takes less space on you computer.

What an autoencoder does is no different. It takes in an input (image etc) and convert it into a more compact format.

How autoencoders do this?

Autoencoders do this by first reduce the size of the input array with layers after layers of processing, and then increase the size of the array also by multiple layers, until the size of the array is the same as the input. The array is then compared to the original input. If the difference is small, then congratulations, you have a good model (because it is able to preserve information). In this case, we take the smallest array along the processing path and say that the array is the most compressed (think in data flowing through the model). We call the compressed vector a latent representation. We separate the model into two parts, the part that processes the input into the compressed input is called encoder, and the part decompressing the compressed input is called decoder.

Why are autoencoders generative models?

The decoder part of an autoencoder is a generative model.

Why? Well, take an autoencoder and cover the encoder. You see that the decoder takes in a lower dimension input and gives a high dimension output that looks like real world examples. That’s basically what every generative model does!