Long Short Term Memory

Note

Long short term memory is often abbreviated as LSTM.

What are LSTMs?

LSTM is a special kind of recurrent layer. A human brain use both long-term memory and short-term memory to remember things, and LSTM is that idea in neural network. Its construct allows some input to be unprocessed by the layer (long term memory) while processing a portion of the input (short term memory).

When to use LSTMs?

Compare to vanilla RNN (the one introduced in the previous section), almost always. Vanilla RNNs are too difficult to train because of gradient issues, while because LSTM allow some input to escape processing, it helps tremendously in that regard.