Embedding Layer

What do embedding layers do?

Embedding layers convert a token (an integer) to a vector (a list of floating point numbers).

When to use embedding layers?

When you want to process text. Text can be converted to integers, but because neural networks don’t directly understand integers (because they are based on gradients, and you can’t compute gradients on integers), you have to use embedding layers to convert those integers into a list of ‘features’. For example, if one of the dimension shows how red the word is, then apple should score a lot higher than banana in that dimension.