Facial expression recognition based on fusion of deep learning and dense SIFT

With the wide application of facial expression recognition in the field of human-computer interaction, accurate and efficient expression recognition methods are of particular important. A hybrid model that combines the convolutional neural network with Dense SIFT features is proposed. The network structure used in the hybrid model is improved in the idea of depth-separable convolutional neural network MobileNet. Based on the separation of channel convolution ( depth convolution)and space convolution (point convolution), the multi-scale convolution kernel is used in the point convolution part of the MobileNet structure, which ensures the diversity and subtleness of the extracted features and is more suitable for facial expression feature extraction, and the introduction of DenseNet network structure ideas improve the performance of the network structure. Using Dense SIFT's 128-dimension descriptors to provide greater advantages for feature descriptions, the improved MobileNet network is integrated with its fully connected layer, and the Eltwise layer is used to compare the elements of the fully connected layer, taking the maximum value to ensure the diversity of features, as well as greater representation. Using this hybrid model on FER2013 and JAFFE face expression data sets, the recognition rate can reach 73.2% and 96.5%.