Most of the current methods of voice translation is based on the conversion of speech into text and back, which often leads to errors and even distortion of the meaning of the spoken phrase. Google has introduced an original technology that can directly convert speech from one language to another while maintaining the tone and frequency of the speaker's speech.
A new translation method called Translatotron uses a network model that treats the user's voice as a spectrogram (visual representation of frequencies) and generates a new spectrogram in the target language. As a result of this operation, the speed and accuracy of translation are significantly increased.
The tool is also able to recognize the features of the timbre and intonation of the speaker and "shift" them to another language, partly simulating the accent and speed of speech of the original. Despite some "robotics", in some cases the computer voice sounds very plausible. You can listen to samples of such translation on the official Google blog page.
Previously, the company introduced the Google Translate feature mimic accents with the support of the pronunciations of various regions. pronunciation by region, and added additional languages to its real-time translation feature. In 2019, Google Assistant voice assistant also received "translator mode" for 26 languages.