Combining image-to-video models with Sync in ComfyUI

Demo summary
The video demonstrates a complex workflow using Krea.ai to transform a shot into a cinematic style before passing the generated video into Sync for final lip-syncing.
Step-by-step
- Select an image-to-image model such as Cadream or Quan Flax
- Apply a prompt to transform the original shot into a cinematic style
- Generate a short video clip based on the new image to establish motion
- Plug the generated video clip into the Sync node
Options
- Use Cadream, Quan Flax, or other current image-to-image models
- Transform the subject into various characters like an alien, pirate, or different persona
Watch out for
- Sync requires a video input rather than a static image to understand face and body motion
Tips
- Use high-quality prompting to achieve a cinematic look that matches specific movie styles
- Ensure the generated video clip captures the necessary motion before attempting the lip-sync
Highlights
“there are no limits”
All demos from “AI lipsync with Sync”
1:220:39Translate and lip-sync video with Sync.soThe creator demonstrates the Sync.so web interface by creating a new project, uploading a video, and pasting translated Spanish text to generate a lip-synced output.Sync.So· AI Lip Sync Generator
2:111:30Advanced voice cloning and obstruction detection in Sync.soThe demo shows how to upload an audio sample for voice cloning and toggle the 'detect obstructions' setting to maintain lip-sync accuracy when a hand covers the mouth.Sync.So· AI Lip Sync Generator
4:460:35Installing and using the Sync custom node in ComfyUISebastian walks through installing the Sync custom node via Git URL in ComfyUI and setting up a workflow with video/audio input nodes and the Sync generate node.ComfyUI· AI Lip Sync Generator
5:291:09Combining image-to-video models with Sync in ComfyUICurrentThe video demonstrates a complex workflow using Krea.ai to transform a shot into a cinematic style before passing the generated video into Sync for final lip-syncing.ComfyUI· AI Lip Sync Generator
7:070:23Multi-speaker lip-sync with 'Detect Active Speaker'The demonstration shows how to enable the 'detect active speaker' setting in Sync to automatically identify and sync only the person currently talking in a multi-person frame.Sync· AI Lip Sync Generator- Watch “AI lipsync with Sync” →
AI Lip Sync Generator
10:070:59Lip-syncing with custom audioThe creator demonstrates importing an audio file into the timeline and using a specific prompt structure to synchronize the character's mouth movements with the audio track.What Dreams Cost
6:310:34Animate talking avatar with LTX 2.3The creator uploads the generated image and audio into the ComfyUI LTX 2.3 workflow and runs the generation to produce a lip-synced video.Prince does AI
14:523:11Lip-syncing audio to images with LTX 2.3The tutorial shows how to upload an MP3 file and a character image to generate a lip-synced video, including a demonstration of the voice isolation subgraph in ComfyUI.AIKnowledge2Go
36:000:39Singing characters with LTX-2 and custom audioThe creator demonstrates how to use LTX-2 to make a character sing by providing a custom audio file and a specific singing prompt.pixaroma
8:381:53Multi-character lip-sync workflow in InfiniteTalkThe tutorial demonstrates the multi-audio workflow, showing how to assign different audio files to specific characters (parallel vs. add types) for a multi-person scene.Atelier Darren
4:460:35Installing and using the Sync custom node in ComfyUISebastian walks through installing the Sync custom node via Git URL in ComfyUI and setting up a workflow with video/audio input nodes and the Sync generate node.Sebastian Kamph
5:291:09Combining image-to-video models with Sync in ComfyUICurrentThe video demonstrates a complex workflow using Krea.ai to transform a shot into a cinematic style before passing the generated video into Sync for final lip-syncing.Sebastian Kamph
5:411:07Configure frame window and motion settings for Infinite TalkThe user demonstrates how to adjust the frame window size and motion frame overlap to balance video smoothness, lip-sync consistency, and VRAM usage.pixaroma
8:411:11Create a multi-person talking video in ComfyUIThe creator demonstrates the multi-talk version of the workflow, showing how to assign separate audio files to two different characters in a single image using the multimodel setting.pixaroma
3:270:37Two-person image to video conversation in ComfyUIThe creator demonstrates how to use the 'add' setting in the ComfyUI node widget to load separate audio files for a back-and-forth conversation between two characters.ComfyUI Studio
4:041:22Parallel audio lip-sync for two charactersThe video shows how to use the 'PAR' setting in the WAN workflow to merge audio tracks for simultaneous or perfectly timed character interactions, including masking two faces in the node.ComfyUI Studio
4:341:29Video-to-video lip sync with Infinite TalkThe creator shows how to use a vid-to-vid workflow to analyze an audio file and automatically synchronize a character's mouth and expressions in an existing video clip.Yaroflasher
ComfyUI