Overview

• RBMs have two layers:
• Some common applications of RBMs are:
• The 'Restricted' in RBM means that neurons within the same layer are not connected to one another
• RBMs are the main block of Deep Belief Networks.
• A Big advantage of RBMs is that they excel when working with unlabelled data.
• RBMs learn patterns and extract important features in data by reconstructing the input.
• RBM is a Generative Model, as opposed to a Discriminative Model:
• RBMs are like Autoencoders (technically they are actually a type of Autoencoder):
• RBMs are stochastic

Mathematical background

• RBM is an Energy Based Model.
• RBM is a Probabilistic Model.

Process of operation

• Forward pass
• Backward pass

Training

• The goal, when training an RBM, is to maximise the likelihood of our data being drawn from the distribution. - This goes hand in hand with minimising the energy function (since the two are inversely related).
• RBM learning algorithms are based on gradient ascent, on the $\log$-likelihood. (ascent as we want to maximise the probability).
• We deduce how much to change weights by using: