ComfyUI: Final video generation with Wan 2.1
ComfyUIDemo summary
The demonstration shows how to use the Wan 2.1 V2V model to combine the generated 3D motion and a reference character image into a final, high-quality AI video animation.
Step-by-step
- Load the enhanced image and the original reference image into the workflow
- Set the dimensions to match the resolution of the motion video
- Set the frame count to match the length of the motion sequence (e.g., 150 frames)
- Bypass the Canny edge node
- Plug DW Pose directly into the control video input
- Disconnect the image caption node to bypass auto-captioning
- Paste your custom prompt directly into the text box for the clip text encoder
- Double-check the video length before generating
Options
- Use Wan 2.1 V2V or Wan 2.1 T2V models
- Adjust width and height manually or match the motion video resolution
- Use Depth Anything V2 for control instead of DW Pose
Watch out for
- If using GGUF models, you must use the correct loader in ComfyUI
- Ensure the generated frame count matches the motion input frames for perfect sync
Tips
- Use DW Pose instead of Canny or Depth when matching a very specific pose
- Bypass auto-captioning (MiniCPM-V) and use manual prompts for highly customized scenes
- Keep reference images in a local folder for easy access during the workflow
- Don't worry about FPS settings as this V2V method is not locked to the model's default 16fps
Highlights
“The settings are pretty straightforward.”
All demos from “Hunyuan Motion 1.0 - This Free And Open Source AI Creates 3D Animation In Seconds!”
- 6:471:07Setting up Hunyuan Motion nodes in ComfyUIThe creator demonstrates how to add the 'load model' and 'motion generate' nodes in ComfyUI, connect them to a 3D preview node, and view a real-time skeleton animation of the generated motion.ComfyUI· AI Animation Generator
- 8:210:59Generating 3D motion from text promptsUsing the Hunyuan Motion 1.0 model in ComfyUI, a text prompt describing a person jumping between buildings is used to generate a specific 3D motion sequence.ComfyUI· AI Animation Generator
- 13:101:04Creating a character scene with DW Pose and Qwen2-VLThe creator shows how to extract a pose from the rendered motion using DW Pose and generate a descriptive caption using Qwen2-VL to prepare for final image generation.ComfyUI· AI Image Generator
- 15:352:08Final video generation with Wan 2.1CurrentThe demonstration shows how to use the Wan 2.1 V2V model to combine the generated 3D motion and a reference character image into a final, high-quality AI video animation.ComfyUI· Video to Video
- Watch “Hunyuan Motion 1.0 - This Free And Open Source AI Creates 3D Animation In Seconds!” →
Video to Video
2:280:44Swap human character for 3D Pixar model in MochaThe creator demonstrates swapping a real person in a video with a 3D Pixar-style character using Mocha, showing the transfer of lip movements and hand gestures.AI Search- 20:110:27Generating final character swap videoThe video shows the K-Sampler process and the final rendered output of a character replacement task within ComfyUI.AI Search
15:344:59Character replacement with Wan AnimateThe creator demonstrates using Wan Animate and SAM 3 to mask a character in a reference video and replace them with a new character from a static image while maintaining the original motion.pixaroma- 21:072:43Cartoon animation with Wan SCAILThe user shows how to animate a cartoon ballerina using a real-person video as a motion reference via the Wan SCAIL workflow in ComfyUI.pixaroma
8:150:26Using Keyframes to guide video generationThe demo shows how to slide an image to a later point in the timeline to use it as a target keyframe, allowing the model to generate the action leading up to that specific visual.What Dreams Cost- 15:352:08Final video generation with Wan 2.1CurrentThe demonstration shows how to use the Wan 2.1 V2V model to combine the generated 3D motion and a reference character image into a final, high-quality AI video animation.Benji’s AI Playground
1:003:01Generate character-consistent video with Wan 2.6 R2VThe creator demonstrates a ComfyUI workflow using the Wan 2.6 API node to transform a 5-second reference video of himself into a new scene while maintaining character consistency and native audio lip-sync.Sebastian Kamph
4:031:00Generating Guide Videos with BFS NodeThe demo shows how to use the BFS Node extension to combine a reference face image and a driving video into a green-screen guide video required for the LTX sampling process.Veteran AI- 5:030:47Injecting Latent Constraints with Add Guide MultiThe video demonstrates using the 'Add Guide Multi' node from the KJ Node extension to inject the guide video constraints and VAE-encoded latents into the sampling pipeline.Veteran AI
1:551:00Adding reference images for character consistencyThe video shows how to use the MSR custom node to input up to four reference images and a background image to maintain character facial consistency across video frames.bigboss97
8:521:04Replacing a dancing character with MochaA concrete demonstration of replacing a woman dancing in a video with a new character using the Mocha workflow in ComfyUI.SOTAI- 12:450:58Replacing a human with an animated werewolfThe video demonstrates replacing a human actor with a stylized werewolf character while maintaining consistent textures and movement.SOTAI