Meta CEO Mark Zuckerberg shared a video in which an engineer who helped develop this tech spoke Hokkien and it was translated to English for Zuckerberg’s as well as audience’s understanding. It is a two-way translation tool – English is converted to Hokkien and Hokkien gets translated to English.
Hokkien is one of 3,500 languages that are spoken and do not have any written system, at least not wide enough to train an AI model. Since, training AI requires large amounts of written text, and there’s not enough written material, how did Meta train its AI?
How Meta trained AI to translate Hokkien
Meta says it developed a variety of methods, including using speech-to-unit translation to translate input speech to a sequence of acoustic sounds, and generated waveforms from them or rely on text from a related language. The related language in this case is Mandarin.
“While the Hokkien translation model is still a work in progress and can translate only one full sentence at a time,” Meta said in a blogpost while announcing that the Facebook-parent company is open-sourcing Hokkien translation models, evaluation datasets and research papers so that others can held and build on the ongoing work.
Meta says that this AI-powered translation tool is part of the company’s Universal Speech Translator project. Under this project, the company is developing new AI methods to allow real-time speech-to-speech translation of many oral languages.
Meta says that a direct speech-to-speech translation approach could lead to much faster, more efficient translation systems by eliminating additional steps such as converting speech to text, translating it, and then generating speech in the target language which are required in most speech translation systems.
