A facial expression is like a gesture which is executed with the facial muscles, and it conveys the emotional state of the subject to the observer. As technology is progressing it is important for the robots to understand human emotions for better communication. We investigate the field of facial expression recognition with deep learning by using Convolutional Neural Network (CNN) algorithm. The proposed framework is trained with FER2013 dataset and tested in real time. Several pooling layers are used in the training part for extracting the features out of the images and haarcascade classifier is used to identify the presence of face in the frame. Our system can recognize five universal emotions of the Facial Action Coding System (FACS) i.e., happy, sad, neutral, surprise and angry which can be identified by the face of a person. It can detect the expressions of multiple faces at the same time. The training accuracy obtained by the system is 73.12%. The real time experiment showed an excellent result.