数字人 Microsoft has announced VASA, which can adjust character states in real time, recognize emotions, and support speech and singing.

Microsoft Research Asia has unveiled the digital human VASA. Its capabilities are very powerful and distinctive. In addition to conventional speech and head movements, it also supports lip movements, facial expressions, and head motion

Core Features

The core of this technology lies in its ability to use a facial latent space to generate facial dynamics and head motion. It can also handle a variety of different conditions, such as gaze direction, head distance, and even emotional offset, and it supports processing photos and audio in different styles—even artistic photos, singing audio, and even non-English speech. It can also disentangle appearance, 3D head pose, and facial dynamics, allowing you to separately control and edit the generated content.

Parameter Adjustment

As shown in the figure, in the sample video, you can adjust the character's orientation, expression, emotional pace, etc., by changing the parameters on the right.



Processing Speed

Even in terms of real-time processing, it performs very well. It can generate 512x512 video frames at up to 45fps, and the startup latency is only a short 170ms!

Release Date

At present, the project has just been announced. The Microsoft team says it still needs some time for adjustment and legal study/review. After complying with these rules, they will launch it, expected this year.


Effect Preview


https://www.microsoft.com/en-us/research/project/vasa-1/  You can learn about the preview video, technical report, and other information in this document.