\n<\/p>\n
Today, we\u2019re announcing a breakthrough in generative AI for speech. We\u2019ve developed Voicebox, a state of the art AI model that can perform speech generation tasks \u2014 like editing, sampling and stylizing \u2014 that it wasn\u2019t specifically trained to do through in-context learning.<\/span><\/p>\n Voicebox can produce high quality audio clips and edit pre-recorded audio \u2014 like removing car horns or a dog barking \u2014 all while preserving the content and style of the audio. The model is also multilingual and can produce speech in six languages.<\/span><\/p>\n In the future, multipurpose generative AI models like Voicebox could give natural-sounding voices to virtual assistants and non-player-characters in the metaverse. They could allow visually impaired people to hear written messages from friends read by AI in their voices, give creators new tools to easily create and edit audio tracks for videos, and much more.<\/span><\/p>\n The versatility of Voicebox enables a variety of tasks, including:<\/span><\/p>\n In-context text-to-speech synthesis: <\/b>Using an audio sample as short as two seconds long, Voicebox can match the audio style and use it for text-to-speech generation.<\/span><\/p>\n Speech editing and noise reduction: <\/b>Voicebox can recreate a portion of speech that\u2019s interrupted by noise or replace misspoken words without having to re-record an entire speech. For example, you can identify a segment of a speech that\u2019s interrupted by a dog barking, crop it, and instruct Voicebox to re-generate that segment \u2013 like an eraser for audio editing.<\/span><\/p>\n Cross-lingual style transfer: <\/b>When given<\/span> a sample of someone\u2019s speech and a passage of text in English, French, German, Spanish, Polish or Portuguese, Voicebox can produce a reading of the text in any of those languages, even when the sample speech and the text are in different languages. This capability could be used in the future to help people communicate in a natural, authentic way even if they don\u2019t speak the same languages.<\/span><\/p>\n Diverse speech sampling<\/b>: Having learned from diverse data, Voicebox can generate speech that is more representative of how people talk in the real world and in the six languages listed above.<\/span><\/p>\n Voicebox is an important step forward in our generative AI research, and we look forward to continuing our exploration in the audio space and seeing how other researchers build on our work.<\/span><\/p>\n Learn more about <\/span>Voicebox<\/span><\/a>.<\/span><\/p>\n<\/p><\/div>\n