Beijing 2022 Winter Olympic Games and winter Paralympic Games are not only a sports event for athletes from all over the world, but also a “wisdom” event enabled by science and technology. In order to make more people feel the charm of the Beijing Winter Olympic Games, recently, in the program “Beijing you are early” of Beijing Satellite TV, the Winter Olympic sign language broadcasting digital people have a new “post”, bringing friendly and natural Winter Olympic sign language broadcasting services to the hearing-impaired people.
What are the technical advantages of digital people Broadcasting in sign language? How to balance speed and accuracy in digital human sign language broadcasting? After the Winter Olympics, what scenarios can this technology be applied in? On the 9th, the reporter walked into Zhipu AI, one of the R & D enterprises of digital people Broadcasting in sign language for the Winter Olympics, and felt the “excellence” of digital people Broadcasting in sign language outside the stadium.
quick step: sign language broadcast faster
According to the data of the second national sample survey of the disabled, there are more than 27 million people with hearing impairment in China. The Beijing Winter Olympics is the first winter Olympics in Chinese history. People with hearing impairment are also looking forward to an in-depth understanding of the competition information and a comprehensive experience of the Winter Olympics.
On February 5, using super large-scale intelligent information model and virtual digital man technology, the Winter Olympic sign language broadcasting digital man created for hearing-impaired people officially appeared on Beijing Satellite TV to bring professional sign language translation and broadcasting of event news during the Winter Olympic Games to the audience.
Zuo Jiaping, partner and senior vice president of Zhipu AI, introduced that the Winter Olympic sign language broadcasting digital human system takes the super large-scale pre training model as the core technology, independently builds a multi-modal limb movement, expression and finger synchronous acquisition system, and uses industry-leading technologies such as cross-modal anthropomorphic generation algorithm and ultra-high-precision realistic digital human, Realize professional sign language translation and broadcasting of event news during the Winter Olympics.
What are the advantages of digital human broadcasting compared with traditional artificial sign language broadcasting? Zhang Peng, chief technology officer of Zhipu AI, said that the biggest advantage of sign language broadcasting digital man is that it is an automatic system, which does not need too much manual intervention and can save a lot of manpower. At the same time, the running speed of the system is close to real-time, so when presenting the effect of sign language broadcasting, it is faster than the traditional manual broadcasting.
profound “knowledge”: richer corpus
In 2018, the national common sign language vocabulary and the national common Braille program were officially released as language and text specifications.
In order to promote and popularize the national general sign language, the Winter Olympic sign language broadcasting digital man system has completed the collection and recording of 8214 general sign languages included in the national general sign language dictionary, and the grammar is subject to the customary playing method of hearing-impaired groups, so as to ensure the accuracy and professionalism of sign language broadcasting results and better serve the hearing-impaired groups.
Due to the lack of perfect sign language corpus data in China, the system R & D personnel, with the support of the Beijing Disabled Persons’ Federation and the Deaf Association of the Beijing Disabled Persons’ Federation, invited more than 40 deaf teachers and sign language experts to provide sign language text transcription and technical guidance, and conduct a large-scale evaluation of hearing-impaired groups, Finally, China’s largest multimodal sign language corpus in line with the national general sign language standard is constructed, with a total scale of more than 100000 words and sentences.
accurate translation: the broadcasting method is more intelligent
Zhang Peng said that compared with the traditional voice AI broadcasting, the biggest difference between the digital people in the Winter Olympic sign language broadcasting is the accuracy of ideographic expression and the intelligibility of expression: the technical characteristics of voice broadcasting mainly focus on the understanding of voice; In the face of hearing-impaired people, sign language broadcasting also needs to use rhythmic gestures, rich and even exaggerated expressions to improve the intelligibility of broadcasting.
After “understanding” the voice, how can digital people express the text more accurately in sign language? It is reported that in order to build an intelligent digital brain that can understand and translate voice and sign language, the Winter Olympic sign language broadcasting digital human system takes the super large-scale pre training model as the core technology, distills the news broadcasting voice, distills the sign language words with highly similar idioms and meanings, and translates them into the word order in line with sign language habits through semantic distillation and sign language translation quick editing model. Finally, the digital brain of sign language can imitate the brain of people with hearing impairment through computer to drive sign language broadcasting.
In addition, in order to achieve high-precision and natural character images and sign language gestures, the R & D team also independently built a multi-modal limb movement, expression and finger synchronous acquisition system. The face acquisition is driven by muscle binding technology. Combined with the industry-leading technologies such as voice recognition and HD video synthesis, it presents a kind and natural Winter Olympic sign language broadcasting service to the hearing-impaired people.
convenient life: wider application scenarios
At present, the digital person of Winter Olympic sign language broadcasting is broadcasting the “Winter Olympic events collection” and “watch the Winter Olympics together” in the program “Beijing Morning” on Beijing Satellite TV. The sign language information broadcasting service reduces the operation cost of Winter Olympic programs and facilitates the way for hearing-impaired people to watch the event reports.
In the future, digital people Broadcasting in sign language are expected to land in public places such as airports, stations and banks to facilitate the life of hearing-impaired people. In addition, the application of digital people in sign language broadcasting will also help promote the promotion of national general sign language, promote the popularization of national general sign language standards, create a barrier free environment for the equal participation of persons with disabilities in social life, and make science and technology more warm.
It is reported that the digital people broadcasting the Winter Olympic sign language are supported by the Beijing Municipal Commission of science and technology and the Beijing Centergate Technologies (Holding) Co.Ltd(000931) Management Committee, jointly built by Zhipu AI, Ling Yunguang and Beijing Radio and television. Professor Jia Jia of Tsinghua University and researcher Chen Yiqiang of the Institute of computing of the Chinese Academy of Sciences participated in the research and development of relevant key technologies, The project also received the help and support of the Beijing Disabled Persons’ Federation and the Deaf Association of the Beijing Disabled Persons’ Federation. (end)
| ___ |
