An Adaptive, Dynamic and Semantic Approach for Understanding of Sign Language based on Convolution Neural Network

Abstract

Sign language has been one of the primary sources when it comes to the communication of people with hearing imparity. Just like every country has its own set of languages which people speak over there, similarly, there are different kinds of sign language which differs with regions and nationality. Many research and studies have been going on concerning the recognition of various kinds of sign language. With the help of sign language, people with hearing imparity can communicate with the rest of the society and convey their message, but the communication is not a two-way effective communication that has led to a barrier between us and the people with hearing imparity. Researchers are constantly trying to develop effective Sign Language Recognition (SLR) System. But all these systems that have been devised to date always come with a drawback and i.e., they have been only limited to isolated sign gestures. The main objective that we tend to achieve while devising this Sign Language Recognition System is that the system should assist in converting the input sign into its corresponding text or speech format automatically so that the rest of the society can effectively communicate with the people having hearing imparity and hence the barrier gets removed. Here in this paper, we propose a Sign Language Recognition System which is based on the Convolution Neural Network (CNN). The main purpose of using CNN over TSK is because it not only classifies and recognizes images but also does all the tasks with high accuracy, unlike the other proposed system. This particular technique will extract the important and useful information of the input sign language while ignoring the rest. Then these signed sequences will be modelled by the proposed mechanism and the output will be thrown in form of text or speech. CNN has been recognized as one of the major branches of Neural Networks. It uses multilayer perceptron which gives the system the ability to recognize and classify similarly as the brain does. Moreover, using this technique over all previous techniques will help us to have the least amount of pre-processing.

A typical Convolution Neural Network has an input layer, multiple internally connected processing layers, and an output layer. All the images need to be trained through all these layers to achieve all the required results.