Facial Expression Recognition Using VGG16 and LSTM

Abstract

Facial expression recognition has made significant progress using deep learning, which has received increasing attention across all fields. Mainly, conventional facial expression recognition systems require constrained datasets for optimal performance, making them unsuitable for use on real-time data. Such real-time sequences limit the efficiency and accuracy of the traditional system. An innovative deep learning framework is proposed in this work that combines dual VGG16 and long short-term memory (LSTM) cells to recognize facial expressions in real-time. Three main aspects of the novel framework are: (i) To enhance each image's edge detail and resolve illumination variances, edge enhancement              pre-processing techniques are utilized; (ii) In order to extract spatial features from                        pre-processed images, the VGG16 model is used, which extracts them quite effectively; (iii) An LSTM layer is used in conjunction with VGG16 for extracting temporal relationships between successive frames along with the spatial feature maps. Comparing the experimental data to existing implementations, facial expression recognition has improved considerably in terms of robustness and accuracy.