Overview

• An Autoencoder (also known as Autoassociator or Diabolo networks) is a type of unsupervised learning algorithm that will find patterns in a dataset by detecting key features.
• Autoencoders are used for tasks that involve:
• The dimensionality reduction that is inherent to most autoencoders has several benefits:
• They find important features by recreating a given input.
• Autoencoders are based on Restricted Boltzmann Machines (in other words RBMs are a type of autoencoder).
• Autoencoders are the building blocks of Deep Belief Networks (DBNs).

Mathematical background

• The mathematical background for the steps performed in Autoencoders is the same as that for Restricted Boltzmann Machines as we essentially use a series of these in both the encoding, and decoding steps.

Process of operation

• Generally speaking, the main goal of Autoencoders is to take unlabelled inputs, encode them, and then try to reconstruct them afterwards.
• The specific encoding/decoding method used, depends on the type of autoencoder we choose; some of which are:

Training

• Autoencoders use back propagation in their learning process.
• Generally speaking, since the goal of an autoencoder (in training) is to reproduce its' input, we calculate loss by comparing the output to the input.
• As mentioned in the 'Process of operation' section above, there are various types of autoencoders. The loss functions associated with each of these are usually subtle variations on these two loss functions that are mentioned here:
• Once loss is calculated, we minimise the error using Gradient Descent.

Citations

G. E. Hinton, R. R. Salakhutdinov, Reducing the Dimensionality of Data with Neural Networks. ScienceMag (2006). (available at https://www.cs.toronto.edu/~hinton/science.pdf).

(n.d.). Autoencoder - Wikipedia. Retrieved from https://en.wikipedia.org/wiki/Autoencoder