A Deep Learning Approach for Gloss Sign Language Translation using Transformer
Mohamed, A., Hefny, H., Amin, M. (2022). A Deep Learning Approach for Gloss Sign Language Translation using Transformer. EKB Journal Management System, 1(2), 1-8. doi: 10.21608/jocc.2022.254979
Ammar Mohamed; Hesham Hefny; mohamed Amin. "A Deep Learning Approach for Gloss Sign Language Translation using Transformer". EKB Journal Management System, 1, 2, 2022, 1-8. doi: 10.21608/jocc.2022.254979
Mohamed, A., Hefny, H., Amin, M. (2022). 'A Deep Learning Approach for Gloss Sign Language Translation using Transformer', EKB Journal Management System, 1(2), pp. 1-8. doi: 10.21608/jocc.2022.254979
Mohamed, A., Hefny, H., Amin, M. A Deep Learning Approach for Gloss Sign Language Translation using Transformer. EKB Journal Management System, 2022; 1(2): 1-8. doi: 10.21608/jocc.2022.254979

A Deep Learning Approach for Gloss Sign Language Translation using Transformer

Journal of Computing and Communication
Article 1, Volume 1, Issue 2, August 2022, Page 1-8  XML PDF (660.32 K)
Document Type: Original Article
DOI: 10.21608/jocc.2022.254979
View on SCiNiTO View on SCiNiTO
Authors
Ammar Mohamed email 1; Hesham Hefny2; mohamed Amin3
1Faculty of Graduate Studies for Statistical Research Cairo University
2Department of Computer Science, Faculty of Graduate Studies for Statistical Research, Cairo University
3Department of Computer Science, Faculty of Graduate Studies for Statistical Research, Cairo University, Cairo, Egypt
Abstract
One of the most recent applications of machine learning is to translate sign language into natural language. Many studies have attempted to classify sign language based on whether it is gesture or facial expression. These efforts, however, ignore genuine sentences' linguistic structure and context. The quality of traditional translation methods is poor, and their underlying models are not salable. They also take a long time to complete. The contribution of this paper is that it suggests utilizing a transformer to perform bidirectional translation using a deep learning approach. The proposed models experiment on the ASLG-PC12 corpus. The experimental results reveal that the proposed models outperform other approaches to the same corpus in both directions of translation, with ROUGE and BLEU  scores of  98.78% and 96.89%, respectively, when translating from text to gloss. Additionally, the results indicate that the model with two layers achieves the best result with ROUGE and BLEU scores of  96.90% and 84.82% when translating from gloss to text.
Keywords
Neural Machine Translation; Sequence to Sequence Model; Sign Language; Deep learning; Transformer
Statistics
Article View: 403
PDF Download: 1,101