ComfyUI: Setting up Wan 2.1 and InfiniteTalk models

Step-by-step

1. Configure the Wan video torch compile settings to enable faster subsequent runs.
2. Set the Wan Video Block Swap 'blocks to swap' setting to 15 to manage VRAM via CPU offloading.
3. Load the Light X2V LoRA to reduce generation steps from 30-50 down to 4-10.
4. Load the InfiniteTalk GGUF model using the Wan multi-talk model loader.
5. Select the single person variant for audio conditioning unless syncing multiple speakers.
6. Load the UMT5 double XL text encoder for prompt processing.
7. Use the Clip Vision Loader to encode the reference image for identity preservation.
8. Configure the Wave 2 vector model for acoustic feature encoding.

Options

• Increase 'blocks to swap' if you encounter CUDA out of memory errors.
• Use the multi-person variant of InfiniteTalk if you need to sync multiple speakers.

Watch out for

• The first run after enabling torch compile will be slow.
• Do not set the Light X2V LoRA strength to 1.0 as it causes color shifts and reduces identity preservation.
• InfiniteTalk is not a standalone generator; it requires Wan 2.1 video to function.
• You must use a video VAE rather than a standard image VAE to preserve motion coherence.

Tips

• Set the Light X2V LoRA strength to 0.8 for the best balance of speed and facial consistency.
• Use the Light X2V LoRA to reduce generation time from several hours to a fraction of that for long videos.

Highlights

“This is where things get technically dense. We have six separate models loading here.”