ComfyUI: Lip-syncing with custom audio

Demo summary
The creator demonstrates importing an audio file into the timeline and using a specific prompt structure to synchronize the character's mouth movements with the audio track.
Step-by-step
- Load an image into the timeline
- Import an audio clip
- Trim the audio clip as needed
- Enter a descriptive prompt for the character
- Toggle custom audio on
- Hit run
Options
- Type the specific words being said in the audio into the prompt to guarantee sync
Watch out for
- Lip sync may fail if the audio is not fully audible
Tips
- Avoid using generic prompts like 'the person speaks' as it often fails to sync
- Use a specific prompt structure found online for better reliability
Highlights
“it works almost all the time”
All demos from “How to use LTX Director - A Free Open Source Tool for Creating LTX 2.3 AI Videos Locally in ComfyUI”
3:070:38Image to Video with LTX DirectorThe creator demonstrates dragging an image into the LTX Director timeline and entering a text prompt to generate a video of a woman waving her hand.ComfyUI· Image to Video
8:150:26Using Keyframes to guide video generationThe demo shows how to slide an image to a later point in the timeline to use it as a target keyframe, allowing the model to generate the action leading up to that specific visual.ComfyUI· Video to Video
10:070:59Lip-syncing with custom audioCurrentThe creator demonstrates importing an audio file into the timeline and using a specific prompt structure to synchronize the character's mouth movements with the audio track.ComfyUI· AI Lip Sync Generator
11:060:26Combining audio, images, and promptsA demonstration of layering a specific action prompt (patting stomach) over a specific audio timestamp to create a fully directed AI scene.ComfyUI· AI Animation Generator- Watch “How to use LTX Director - A Free Open Source Tool for Creating LTX 2.3 AI Videos Locally in ComfyUI” →
AI Lip Sync Generator
14:523:11Lip-syncing audio to images with LTX 2.3The tutorial shows how to upload an MP3 file and a character image to generate a lip-synced video, including a demonstration of the voice isolation subgraph in ComfyUI.AIKnowledge2Go
36:000:39Singing characters with LTX-2 and custom audioThe creator demonstrates how to use LTX-2 to make a character sing by providing a custom audio file and a specific singing prompt.pixaroma
10:070:59Lip-syncing with custom audioCurrentThe creator demonstrates importing an audio file into the timeline and using a specific prompt structure to synchronize the character's mouth movements with the audio track.What Dreams Cost
5:411:07Configure frame window and motion settings for Infinite TalkThe user demonstrates how to adjust the frame window size and motion frame overlap to balance video smoothness, lip-sync consistency, and VRAM usage.pixaroma
8:411:11Create a multi-person talking video in ComfyUIThe creator demonstrates the multi-talk version of the workflow, showing how to assign separate audio files to two different characters in a single image using the multimodel setting.pixaroma
3:270:37Two-person image to video conversation in ComfyUIThe creator demonstrates how to use the 'add' setting in the ComfyUI node widget to load separate audio files for a back-and-forth conversation between two characters.ComfyUI Studio
4:041:22Parallel audio lip-sync for two charactersThe video shows how to use the 'PAR' setting in the WAN workflow to merge audio tracks for simultaneous or perfectly timed character interactions, including masking two faces in the node.ComfyUI Studio
4:460:35Installing and using the Sync custom node in ComfyUISebastian walks through installing the Sync custom node via Git URL in ComfyUI and setting up a workflow with video/audio input nodes and the Sync generate node.Sebastian Kamph
4:341:29Video-to-video lip sync with Infinite TalkThe creator shows how to use a vid-to-vid workflow to analyze an audio file and automatically synchronize a character's mouth and expressions in an existing video clip.Yaroflasher
8:381:53Multi-character lip-sync workflow in InfiniteTalkThe tutorial demonstrates the multi-audio workflow, showing how to assign different audio files to specific characters (parallel vs. add types) for a multi-person scene.Atelier Darren
6:310:34Animate talking avatar with LTX 2.3The creator uploads the generated image and audio into the ComfyUI LTX 2.3 workflow and runs the generation to produce a lip-synced video.Prince does AI
ComfyUI