Multi Layer Perceptron with Tensorflow

Hi. We listen about ML i.e Machine Learning everywhere. But what the hell is MLP and why should we care about it anyway? Well, MLP or Multi Layer Perceptron is an architecture we use in building neural network.

There are 3 most common neural network architectures every Deep Learning practitioner must be aware of

1. Feed Forward Neural Network
2. Recurrent Neural Network
3. Symmetrically Connected Networks

Multi Layer Perceptron is a class of Feed Forward Neural Network . In Feed Forward Neural Network, the flow of data is from input nodes to output nodes , that is why they are called Feed forward.

MLP consists of three layers of nodes : input layer, hidden layer and output layer.

input_hidden_layer = tf.layers.dense(x,n_hidden1, activation = tf.nn.relu)

hidden_layer = tf.layers.dense(input_layer, n_hidden2, activation = tf.nn.relu)

output_layer = tf.layers.dense(hidden2, n_outputs, name = “outputs”)

x : input data

n_hidden1: weights initialized at input layer

n_hidden2 : weights initialized at hidden layer

n_outputs : weights at output layer

The above snippets use Tensorflow library to perform MLP. Let’s go through the above codes one by one. The first layer i.e input_hidden_layer takes input data, multiply it with the weights present at input layer i.e n_hidden1 and finally perform activation function to give the output which can be treated as input for the next layer. The most popular choice of activation function is the rectified linear unit or relU which has the formula:

h(x) = max(0,x)

The second layer i.e hidden layer is same as the first layer except this layer takes the input generated from the previous layer perform some transformations on it using activation function and weights initialized at that layer. In this example we are using two hidden layers but it could be any number you want more than one.

The last layer i.e output_layer turns the tensor size to expected output size. The weights at this layer should be initialized in a way that it transforms the tensors to the expected final output shape.

After the MLP layer has been designed. Our next step is to back propagate to each layer and modify weights to reduce the mean squared difference between the expected output and predicted output. We do this by first calculating the loss and then using an optimizer that will optimize the weights to reduce loss.

Let’s see some sample code for this

xentropy = tf.nn.sparse_softmax_cross_entropy_with_logits(labels = y, logits = logits)

loss = tf.reduce_mean(xentropy, name =”loss”)

Here I’ve used the sparse_softmax_cross_entropy_with_logits to measure the probability error as this function computes error on sparse labels instead of converting them with one_hot_encoding.

optimizer = tf.train.GradientDescentOptimizer(learning_rate)

opt = optimizer.minimize(loss)

Finally we are going to optimize gradients or in simpler terms previously initialized weights using GradientDescentOptimizer function.

Multi Layer Perceptron is the commonest architecture used in building a neural network. The complete code can be found here.

If you like the tutorial please give it a clap. Thank You :)