Autoencoders

Squeeze 144 pixels through two numbers and back. Train it in your browser, watch it discover where the blob sits without being told, then drag a point around the two-number space and see the decoder paint the picture it stands for.

Builds onPrincipal Component Analysis Backpropagation

Trained in your browser · 144 → 32 → 2 → 32 → 144 · 9,554 numbers

Training the autoencoder in your browser (3,000 steps)… 0%

What the bottleneck does

Encoder: 144 → 32 → 2. Decoder: 2 → 32 → 144. The only target is the input itself.
Code: the 2 numbers in the middle. Nothing says what they mean; training lays them out by what varies.
Versus PCA: PCA rebuilds a picture as the average plus a mix of fixed pictures, so it cannot slide a blob. The tanh layers let the code bend.
Every code decodes to some picture, which makes the decoder a generator. Variational autoencoders train the code space to have no holes.

A common misconception

“The picture halfway between two others is their average, pixel by pixel.”

Measuring once the autoencoder has trained…

Check yourself

Question 1 of 3

An autoencoder is trained to make its output equal its input. Why does it not simply learn to copy?

DSAverse

Autoencoders

See also

Autoencoders

See also