LSTM and GRU

Gates are numbers between 0 and 1 that a network learns to set: how much to forget, how much to write. Watch trained networks hold one number while noise streams past — the plain RNN drifts, the gated ones hold, and their real gate values show how.