Real-Time Facial Expression Recognition and Speech Tran-scripts over an on-premise Video Conference Application | ||||
| International Journal of Telecommunications | ||||
| Volume 02, Issue 02 - Serial Number 3, December 2022, Page 1-14 PDF (1.84 MB) | ||||
| Document Type: Original Article | ||||
| DOI: 10.21608/ijt.2022.266291 | ||||
 View on SCiNiTO | ||||
| Authors | ||||
S Eltenahy  ; Nihall Areed; Marwa obayya; Fahmi Khalifa
| ||||
| Mansoura University Electronics and Communication Engineering Department, Faculty of Engi-neering, Mansoura University | ||||
| Abstract | ||||
| Since Covid-19 pandemic outbreak, organizations and individuals have had to use vid-eo conference applications increasingly. However, the commercial video conference applications are expensive, and feature limited. This paper discusses how to enable organizations to host on-premise video conference applications. Then, it explores assisting organization’s stakeholders with making decisions based on facial expressions of video conference attendees. Moreover, it facili-tates transcribing speech into text to enable deaf persons to participate in online conferences. Technologies and tools used in addressing these challenges respectively are: (i) Web Real Time Communication (WebRTC) project, (ii) Tensorflow.js library, (iii) and Web Speech Application Programming Interface (API). This paper depends on integration between a collection of technol-ogies, libraries, standards, and protocols. Most of them can be managed using JavaScript frame-work. Hence, load of the performance is distributed on each client-side device. The proposed on-premise video conference application has been enhanced through including facial expression recognition with 66% high accuracy while the speech-into-text feature with Word Error Rates (WER) are 0 and 0.12 for British English and Egyptian Arabic, respectively | ||||
| Keywords | ||||
| WebRTC; Video conferencing; Facial Expression Recognition; Speech Recog-nition; Computer Vision; ML; TensorFlow.js; OpenVidu; Speech-to-Text | ||||
|
Statistics Article View: 1,132 PDF Download: 887 |
||||
