基于全局注意力机制和LSTM的连续手语识别算法

Continuous Sign Language Recognition Algorithm Based on Global Attention Mechanism and LSTM

中文关键词: 手语识别  特征提取  全局注意力机制  LSTM

英文关键词: sign language recognition  feature extraction  global attention mechanism  LSTM

中文摘要:

      目的 为提高连续手语识别准确率，缓解听障人群与非听障人群的沟通障碍。方法 提出了基于全局注意力机制和LSTM的连续手语识别算法。通过帧间差分法对视频数据进行预处理，消除视频冗余帧，借助ResNet网络提取特征序列。通过注意力机制加权，获得全局手语状态特征，并利用LSTM进行时序分析，形成一种基于全局注意力机制和LSTM的连续手语识别算法，实现连续手语识别。结果 实验结果表明，该算法在中文连续手语数据集CSL上的平均识别率为90.08%，平均词错误率为41.2%，与5种算法相比，该方法在识别准确率与翻译性能上具有优势。结论 基于全局注意力机制和LSTM的连续手语识别算法实现了连续手语识别，并且具有较好的识别效果及翻译性能，对促进听障人群无障碍融入社会方面具有积极的意义。

英文摘要:

      To improve the continuous sign language recognition accuracy and alleviate the communication barrier between hearing-impaired people and non hearing-impaired people, this paper proposed the continuous sign language recognition algorithm based on Global Attention Mechanism and LSTM (CSLR-GAML). The video data is preprocessed by applying inter-frame difference to eliminate redundant video frames, and then the feature sequences of the key frames are extracted by using ResNet. After that, the attention mechanism is used to update the network parameters, which is capable of obtain the global feature of sign language, and then the LSTM is employed to finish the timing sequence analysis. Finally, use the Chinese continuous sign language data set CSL to check algorithm performance. And the experimental results show that the average recognition accuracy of the proposed algorithm is 90.08%, and the average word error rate is 41.2%. Compared CSLR-GAML with other Five algorithms, the proposed CSLR-GAML has advantages in recognition accuracy and translation performance. The sontinuous sign language recognition algorithm based on Global Attention Mechanism and LSTM realizes continuous sign language recognition, and has good recognition effect and translation performance. It is of positive significance to promote the barrier free integration of hearing-impaired people into society.