Rough Sets theory with Deep Learning for Tracking in Natural Interaction with Deaf

Authorsمحمد ابراهیمی,حسین ابراهیم پور کومله
JournalJournal of Information Systems and Telecommunication
Page number211
Volume number10
IFثبت نشده
Paper TypeFull Paper
Published At2022-08-03
Journal GradeScientific - research
Journal TypeElectronic
Journal CountryIran, Islamic Republic Of
Journal IndexSCOPUS ,ISC

Abstract

Sign languages commonly serve as an alternative or complementary mode of human communication Tracking is one of the most fundamental problems in computer vision, and use in a long list of applications such as sign languages recognition. Despite great advances in recent years, tracking remains challenging due to many factors including occlusion, scale variation, etc. The mistake detecting of head or left hand instead of right hand in overlapping are, modes like this, and due to the uncertainty of the hand area over the deaf news video frames; we proposed two methods: first, tracking using particle filter and second tracking using the idea of the rough set theory in granular information with deep neural network. We proposed the method for Combination the Rough Set with Deep Neural Network and used for in Hand/Head Tracking in Video Signal DeafNews. We develop a tracking system for Deaf News. We used rough set theory to increase the accuracy of skin segmentation in video signal. Using deep neural network, we extracted inherent relationships available in the frame pixels and generalized the achieved features to tracking. The system proposed is tested on the 33 of Deaf News with 100 different words and 1927 video files for words then recall, MOTA and MOTP values are obtained.

tags: Natural interaction with deaf; Machine vision; Persian deaf news Hand Tracking; Sign language; Rough sets theory; Deep Learning.