Autoencoders

Autoencoders (AE) are machines that encode inputs into a compact latent space.

Notation: dot ($\cdot$)

We use a single vertically centered dot, i.e., $\cdot$, to indicate that the function or machine can take in arguments.

A simple autoencoder can be achieved using two neural nets, e.g.,

$$\begin{array}{}\text{(1)}& h& =g(b+wx)\text{(2)}& \hat{x}& =\sigma (c+vh),\end{array}$$

where in this simple example,

• $g(b+w\cdot )$ is the encoder, and
• $\sigma (c+v\cdot )$ is the decoder.

For binary labels, we can use a simple cross entropy as the loss.

Code

See Lippe¹.

---

1. Lippe P. Tutorial 9: Deep Autoencoders — UvA DL Notebooks v1.1 documentation. In: UvA Deep Learning Tutorials [Internet]. [cited 20 Sep 2021]. Available: https://uvadlc-notebooks.readthedocs.io/en/latest/tutorial_notebooks/tutorial9/AE_CIFAR10.html ↩

---

Contributors: LM