Sign Language Recognition Using Deep Learning: Advancements and Challenges
Article Sidebar
Main Article Content
Abstract: Sign language recognition (SLR) has arisen as a major area of research in recent years, attempting to bridge the communication gap between the deaf and hard-of-hearing community and the hearing world. This research study addresses the construction and implementation of a manual alphabet recognition system utilising deep learning techniques, notably convolutional neural networks (CNNs). The work focuses on establishing an efficient and accurate system for converting Nigerian Sign Language manual alphabets into text. By integrating computer vision and machine learning methods, the proposed system seeks to overcome the communication gap between deaf and hearing individuals. The paper explains the technique adopted, including data collection, preprocessing, model architecture, and deployment using web-based tools. The system achieves a 95% success rate in recognizing static hand motions, proving its potential for real-world applications. However, issues in identifying dynamic motions and generalizing across varied user populations are observed. The report finishes with recommendations for future research, emphasizing the need for combining temporal analysis and expanding the system's capabilities to word and phrase recognition.
Downloads
Downloads
References
Alaftekin, M., Pacal, I. & Cicek, K. (2024) Real-time sign language recognition based on YOLO algorithm. Neural Computing & Applications 36,7609–7624 https://doi.org/10.1007/s00521-024-09503-6 DOI: https://doi.org/10.1007/s00521-024-09503-6
Asonye, E. I., Emma-Asonye, E., & Edward, M. (2018). Deaf in Nigeria: A Preliminary Survey of Isolated Deaf Communities. Sage Open, 8(2). https://doi.org/10.1177/2158244018786538 DOI: https://doi.org/10.1177/2158244018786538
Asonye, Emmanuel & Emma-Asonye, Ezinne & Edward, Mary. (2020). Linguistic Genocide against Development of Indigenous Signed Languages in Africa.
Cohen, S. (2020). Artificial intelligence and deep learning in pathology. Elsevier Health Sciences. https://doi.org/10.1016/C2018-0-02465-2 DOI: https://doi.org/10.1016/C2018-0-02465-2
Dabwan باسل دبوان, Basel & Jadhav, Mukti & Abosaq, Hamad & Olayah, Fekry & Yami, Mohammed & Ali, Yahya. (2024). Real-time System for Translating American Sign Language to Text Using Robust Techniques.1-6.10.1109/ICRASET59632.2023.10420110 DOI: https://doi.org/10.1109/ICRASET59632.2023.10420110
Deshpande, A., Shriwas, A., Deshmukh, V.J., & Kale, S. (2023). Sign Language Recognition System using CNN. 2023 International Conference on Intelligent and Innovative Technologies in Computing, Electrical and Electronics (IITCEE), 906-911. DOI: https://doi.org/10.1109/IITCEE57236.2023.10091051
Eleweke, C. (2002). A Review of Issues in Deaf Education Under Nigeria's 6-3-3-4 Education System. Journal of deaf studies and deaf education. 7. 74-82. 10.1093/deafed/7.1.74. DOI: https://doi.org/10.1093/deafed/7.1.74
Gordon, R. G., Grimes, B. F., & Summer Institute of Linguistics. (2005). Ethnologue: Languages of the world (15th ed.). SIL International.
Greenberg, S., Blight, J., & Wong, A. Colour-based Gesture Recognition for American Sign Language via Hidden Markov Models. University of Waterloo, Canada.
Joudaki, S., Mohamad, D., Saba, T., Rehman, A., Al-Rodhaan, M., & Al-Dhelaan, A. (2014). Vision-based sign language classification: A directional review. IETE Technical Review, 31(5), 383-402. https://doi.org/10.1080/02564602.2014.961576 DOI: https://doi.org/10.1080/02564602.2014.961576
Karbasi, M., Shah, A., & Landani, Z. (2015). An analysis of vision-based Malaysian sign: A review. International Journal of Advanced Research in Science, Engineering and Technology, 2(1), 395- 399.
Lillo-Martin, D., & Sandler, W. (2006). Sign language and linguistic universals. Cambridge University Press. Merriam-Webster. (n.d.). Manual alphabet. In Merriam-Webster.com dictionary. Retrieved October 22, 2024, from https://www.merriam-webster.com/dictionary/manual%20alphabet
Padden, C. (2003). How the alphabet came to be used in a sign language. Sign Language Studies, 4(1), 10-33. https://doi.org/10.1353/sls.2003.0026 DOI: https://doi.org/10.1353/sls.2003.0026
Pathan, R.K., Biswas, M. and Yasmin, S. (2023). Sign language recognition using the fusion of image and hand landmarks through multi-headed convolutional neural network. Sci Rep 13, 16975 (2023). https://doi.org/10.1038/s41598-023-43852-x DOI: https://doi.org/10.1038/s41598-023-43852-x
Paulraj, M. P., Yaacob, S., Azalan, M. S. Z., & Palaniappan, R. (2010). A phoneme based sign language recognition system using skin color segmentation. 6th International Colloquium on Signal Processing & Its Applications (CSPA), 1-5. IEEE. https://doi.org/10.1109/CSPA.2010.5545291 DOI: https://doi.org/10.1109/CSPA.2010.5545253
Oguntimilehin, A., & Balogun, K. (2024). Real-Time Sign Language Fingerspelling Recognition using Convolutional Neural Network. The International Arab Journal of Information Technology, 21(1). https://doi.org/10.34028/iajit/21/1/14 DOI: https://doi.org/10.34028/iajit/21/1/14
Shin, Jungpil & Matsuoka, Akitaka & Hasan, Md. Al & Srizon, Azmain. (2021). American Sign Language Alphabet Recognition by Extracting Feature from Hand Pose Estimation. Sensors (Basel, Switzerland). 21. 10.3390/s21175856. DOI: https://doi.org/10.3390/s21175856
Simon, C. (1982). International hand alphabet charts (2nd ed.). National Association of the Deaf.
Swee, T. T., Ariff, A. K., Salleh, S. H., Seng, S. K., & Huat, L. S. (2007). Wireless data gloves Malay sign language recognition system. 6th International Conference on Information, Communications & Signal Processing, 1-4. IEEE. https://doi.org/10.1109/ICICS.2007.4449599 DOI: https://doi.org/10.1109/ICICS.2007.4449599
World Health Organization. (2024). Deafness and hearing loss. https://www.who.int/news-room/fact-sheets/detail/deafness- and-hearing-loss
Zhang, Yanqiong & Jiang, Xianwei. (2024). Recent Advances on Deep Learning for Sign Language Recognition. Computer Modeling in Engineering & Sciences. 139. 1-10.10.32604/cmes.2023.045731. DOI: https://doi.org/10.32604/cmes.2023.045731
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles published in our journal are licensed under CC-BY 4.0, which permits authors to retain copyright of their work. This license allows for unrestricted use, sharing, and reproduction of the articles, provided that proper credit is given to the original authors and the source.