ComfyUI: Running MultiTalk on RunPod

Step-by-step

1. Drag and drop the 720p long context high quality workflow into ComfyUI
2. Upload your demo image and audio file
3. Set the width and height to match your image (e.g., 720x1280)
4. Calculate and enter the number of frames based on audio length (25 frames per second)
5. Enter the text prompt describing the character's action
6. Set 'blocks to swap' to 0 for high VRAM machines or 40 for low VRAM machines
7. Click the Run button at the bottom of the interface
8. Monitor the generation progress via the terminal or nvitop
9. Right-click the result and select Save Preview to download the video

Options

• Enable 'tiled VAE' or 'all tiling' if you encounter out-of-memory errors
• Bypass the block swapping node on high VRAM machines to speed up model loading
• Download the entire output folder as an archive from the workspace file browser

Watch out for

• The number of frames is not automatically set; you must manually calculate it based on audio duration
• Initial model loading on RunPod can take a significant amount of time
• High-quality processing requires 10 steps and significant VRAM (up to 35GB for 720p)

Tips

• Use 25 frames for every one second of audio
• Monitor nvitop to verify the GPU is being fully utilized (e.g., checking watt usage and VRAM)
• Check the terminal status to see the exact duration of the uploaded audio for more accurate frame calculation

Highlights

“the machine is ready and set... the rest of the usage is exactly same as in the Windows tutorial part.”