KAZAKH SIGN LANGUAGE INTERPRETER USING DEEP LEARNING

Аннотация

Abstract. Everyone can see, listen, and respond to his/her surroundings. There are some people who do not see, listen, and respond to his/her surroundings. Such individuals are mainly the dumb and deaf people. These people depend on sign language to interact with others. However, communication with normal people is a major problem for them because majority of normal people unable to understand their sign language. This will cause a problem for the dumb and deaf people to communicate with others, particularly when they are in social, educational, and work environments. This proposed system was developed in order to assist the hearing or speech impaired people to communicate with normal people. The main goal of this project is to develop sign language translation system that can translate the sign language into text using Convolutional Neural Networks. This uses the property of convolution, mainly devised for analyzing visual imagery. Segmented RGB hand gestures were fed to three layered Convolutional Neural Networks for training and testing in real time. The image dataset, for each gesture, was created using simple image of the hand taken with a personal device such as a laptop webcam.