Github ByteDance OmniHuman image-to-video

OmniHuman-1: Rethinking the Scaling of Single-Stage Conditional Human Animation Models
https://github.com/omnihuman-lab/omnihuman-lab.github.io

Generated Videos
OmniHuman supports a wide range of visual and audio styles and can generate realistic human videos with arbitrary aspect ratios and body proportions (portrait, half-body, and full-body unified). The realism comes from a combination of aspects such as motion, lighting, and texture details.
*Please note that generating all the results on this page requires only any single image and audio, except for demos showing video and combined driving signals. To keep the layout tidy, we omit the display of reference images; in most cases, the reference image is the first frame of the generated video. If you need comparisons or more information, please feel free to contact us.


OmniHuman can support audio-driven inputs in any aspect ratio. It significantly improves gesture handling, which is a challenge for existing methods, and produces highly realistic results. The audio and images for some test cases come from link1, link2, link3, and link4.

Diversity
In terms of input diversity, OmniHuman supports cartoons, artificial objects, animals, and challenging poses, ensuring that motion characteristics match the unique features of each style.



Half-body with hands
Here, we also provide additional examples specifically showcasing hand gestures. Some input images and audio come from TED, Pexels, and AIGC.