Perceptron Learning Algorithm


Perceptron is a learning algorithm of machine learning used as a binary classifier,which is used to classify whether a input belongs to a particular group or not.It is a type of linear classifier in which prediction is based on linear predictor function combine to set of weight with feature vector.

Concept behind the Algorithm:-

We define a activation function  ϕ(z) that takes linear combination of certain input x and corresponding weight vector w,where z is the so called net input z.

z=  {\displaystyle \sum _{i=1}^{m}w_{i}x_{i}} + bias = w^T.x

=> ϕ(z) =1 if z >=0 else -1 otherwise.

In perceptron,in each iteration the weight are updated using:

w = w + learning_rate * (expected – predicted) * x

following is the diagram of working of  ϕ(z):-


perceptron rule can be summarized in following steps:

1.Initialize the weights to 0 or small random number.

2.For each training smaple { x }^{ (i) }   perform the following steps:-

a)compute the output value \widehat { y } .

b)Update the weights.

Here, the output value is the class label predicted by the unit step function that we
defined earlier, and the simultaneous update of each weight { w }_{ j } in the weight vector w can be more formally written as:

{ w }_{ j }\quad =\quad { w }_{ j }\quad +\triangle { w }_{ j }

The value of \triangle { w }_{ j } ,which is used to update the weight { w }_{ j } ,is calculated by the perceptron learning rule:

\triangle { w }_{ j }\quad =\quad \eta \left( { y }^{ (i) }-{ \widehat { y } }^{ (i) } \right) { x }_{ j }^{ (i) }

Here \quad \eta is the learning rate which generally lie between 0.0 and 1.0.{ y }^{ (i) } is the true class label of  { i}^{ th } is the predicted class label.It is important to note that all weight in the weight vector are being updated simultaneously.

It is important to note that the convergence of the perceptron is only guaranteed if
the two classes are linearly separable and the learning rate is sufficiently small. If the
two classes can’t be separated by a linear decision boundary, we can set a maximum
number of passes over the training dataset (epochs) and/or a threshold for the
number of tolerated misclassifications—the perceptron would never stop updating
the weights otherwise.

2017-09-11 (1)

Lets summarize the whole thing that we discuss above in the form of figure:-

2017-09-11 (2)


I’m a python lover,so i do most of a coding part in python.following are some function and definition which i defined in perceptron class:

parameters are:-

eta : learning rate

n_iter : no of passes over the training data

Attribute are:-

w : weights after fitting

error = no. of misclassification in every pass


defining the fit function

def fit(X,y):

w = np.zeros(1+X.shape[1])
errors = []

for _ in range(n_iter):
error = 0
for xi,target in zip(X,y):
update = eta*(target-self.predict(xi))
w[1:] += update*xi
w[0] += update
errors += int(update != 0.0)

def net_input(X):
“””Calculate net input”””

def predict(X):
“””Return class label after unit step”””
return np.where(net_input(X) >=0.0,1,-1)

Using the above definition in perceptron class we can make a object of perceptron model.

Now lets train the perceptron model on iris data sets.

Iris data sets consists of 3 different types of irises’ (Setosa, Versicolour, and Virginica) petal and sepal length, stored in a 150×4 numpy.ndarray

The rows being the samples and the columns being: Sepal Length, Sepal Width, Petal Length and Petal Width.


a)setosa                                         b)versicolor                                 c)virginica


we will try to classify any one of flower with the rest.

lets visulize the whole dataset in scatter plot.


considering sepal lenght and petal length we can classify setosa and versicolor from above diagram.

following is the scatter plot taking petal and sepal lenght as feature:-



Taking eta = 0.01 and n_iter = 10,lets find out the number of missclassification in each iteration:-


We can see from above graph that after the sixth iteration,graph started converging towards classifying the training datasets perfectly.


following is how graph changed after 5th iteration:-

for the actual python code for above discussed things,click below:-

perceptron learning algorithm