ComfyUI: Video-to-video lip sync with Infinite Talk

Demo summary
The creator shows how to use a vid-to-vid workflow to analyze an audio file and automatically synchronize a character's mouth and expressions in an existing video clip.
Step-by-step
- Select the Infinite Talk vid-to-vid workflow in ComfyUI
- Upload the original video clip containing the character performance
- Upload the separate audio file containing the vocal phrase
- Run the workflow to analyze the audio and synchronize the character's mouth and expressions
Options
- Apply the workflow to live-action footage
- Apply the workflow to cartoon or animated characters
- Use footage with varying camera angles including close-ups, medium shots, and wide frames
Tips
- Use this workflow for dynamic edits where the camera moves or the character turns, as the synchronization remains solid
Highlights
“It really does work like a charm.”
All demos from “Stop Paying: 10 Free AI Workflows Better Than Paid”
1:110:46Inpainting face swap with ComfyUIThe creator demonstrates an inpainting face swap workflow in ComfyUI by loading a reference and target image to achieve a high-resemblance result on a close-up shot.ComfyUI· AI Face Swap Generator
3:201:14Animate static images with Infinite Talk V2A demonstration of the Infinite Talk V2 workflow where a static thumbnail image is paired with an audio track to create a lip-synced, animated talking character.ComfyUI· AI Lip Sync Generator
4:341:29Video-to-video lip sync with Infinite TalkCurrentThe creator shows how to use a vid-to-vid workflow to analyze an audio file and automatically synchronize a character's mouth and expressions in an existing video clip.ComfyUI· AI Lip Sync Generator
6:130:58AI outfit and clothing changeUsing a specific ComfyUI workflow, the creator demonstrates replacing clothing on a model while maintaining the original body shape, pose, and identity.ComfyUI· AI Clothing Removal
8:230:24Change hairstyles with AIA demonstration of using an outfit change system to apply a new haircut to a character by using a hairstyle image as a reference.ComfyUI· AI Inpainting
8:470:24AI background replacementThe creator shows how to swap the environment behind a subject while keeping the person and pose consistent using a background reference image.ComfyUI· AI Background Generator
12:511:32Image-to-video generation with SoraA walkthrough of using the Sora 2 workflow to convert a storyboard frame into a 10-second cinematic video clip using a text prompt.ComfyUI· Image to Video
15:031:04Apply camera movement from reference videoThe creator demonstrates copying camera motion and pose dynamics from a reference video (like a dance clip) and applying it to a still character image.ComfyUI· Video to Video
16:071:11Replace character face in generated videoA demonstration of a character replacement pass that swaps a face in a Sora-generated video with a specific reference photo to maintain identity consistency.ComfyUI· AI Face Swap Video- Watch “Stop Paying: 10 Free AI Workflows Better Than Paid” →
AI Lip Sync Generator
14:523:11Lip-syncing audio to images with LTX 2.3The tutorial shows how to upload an MP3 file and a character image to generate a lip-synced video, including a demonstration of the voice isolation subgraph in ComfyUI.AIKnowledge2Go
36:000:39Singing characters with LTX-2 and custom audioThe creator demonstrates how to use LTX-2 to make a character sing by providing a custom audio file and a specific singing prompt.pixaroma
10:070:59Lip-syncing with custom audioThe creator demonstrates importing an audio file into the timeline and using a specific prompt structure to synchronize the character's mouth movements with the audio track.What Dreams Cost
5:411:07Configure frame window and motion settings for Infinite TalkThe user demonstrates how to adjust the frame window size and motion frame overlap to balance video smoothness, lip-sync consistency, and VRAM usage.pixaroma
8:411:11Create a multi-person talking video in ComfyUIThe creator demonstrates the multi-talk version of the workflow, showing how to assign separate audio files to two different characters in a single image using the multimodel setting.pixaroma
3:270:37Two-person image to video conversation in ComfyUIThe creator demonstrates how to use the 'add' setting in the ComfyUI node widget to load separate audio files for a back-and-forth conversation between two characters.ComfyUI Studio
4:041:22Parallel audio lip-sync for two charactersThe video shows how to use the 'PAR' setting in the WAN workflow to merge audio tracks for simultaneous or perfectly timed character interactions, including masking two faces in the node.ComfyUI Studio
5:420:22Video-to-video lip-sync with WAN workflowThe demonstration shows how to take an existing video and sync the lip movements to a new audio file using the ComfyUI workflow.ComfyUI Studio
4:460:35Installing and using the Sync custom node in ComfyUISebastian walks through installing the Sync custom node via Git URL in ComfyUI and setting up a workflow with video/audio input nodes and the Sync generate node.Sebastian Kamph
5:291:09Combining image-to-video models with Sync in ComfyUIThe video demonstrates a complex workflow using Krea.ai to transform a shot into a cinematic style before passing the generated video into Sync for final lip-syncing.Sebastian Kamph
3:201:14Animate static images with Infinite Talk V2A demonstration of the Infinite Talk V2 workflow where a static thumbnail image is paired with an audio track to create a lip-synced, animated talking character.Yaroflasher
4:341:29Video-to-video lip sync with Infinite TalkCurrentThe creator shows how to use a vid-to-vid workflow to analyze an audio file and automatically synchronize a character's mouth and expressions in an existing video clip.Yaroflasher
ComfyUI