demobook

ComfyUI: Singing characters with LTX-2 and custom audio

Demo summary

The creator demonstrates how to use LTX-2 to make a character sing by providing a custom audio file and a specific singing prompt.

Step-by-step

  1. Download the required models into the folders specified in the workflow notes
  2. Load an image into the image loader node
  3. Load a singing or talking audio file into the audio loader node
  4. Enter a prompt describing the character's action, such as 'a woman is singing'
  5. Play the audio file to determine its length
  6. Set the duration value in the workflow to match the length of the audio
  7. Run the workflow to generate the result in the output node

Options

  • Use either singing or talking audio files
  • Add extra descriptive details to the prompt regarding character actions

Watch out for

  • You must download specific models into the correct folders as indicated by the workflow notes before starting

All demos from “ComfyUI Video Models: InfiniteTalk + Wan 2.2 + SCAIL + LTX-2 (Ep06)

  1. 5:244:57Image-to-video generation with Wan 2.2The user demonstrates how to use the Wan 2.2 GGUF model in ComfyUI to animate a static image of a woman based on a detailed text prompt.ComfyUIImage to Video
  2. 13:071:21Applying LoRA to Wan 2.2 video generationThe video shows how to integrate a LoRA (Low-Rank Adaptation) into a Wan 2.2 workflow to achieve specific cinematic movements like a face zoom.ComfyUIAI Animation Generator
  3. 15:344:59Character replacement with Wan AnimateThe creator demonstrates using Wan Animate and SAM 3 to mask a character in a reference video and replace them with a new character from a static image while maintaining the original motion.ComfyUIVideo to Video
  4. 21:072:43Cartoon animation with Wan SCAILThe user shows how to animate a cartoon ballerina using a real-person video as a motion reference via the Wan SCAIL workflow in ComfyUI.ComfyUIVideo to Video
  5. 25:063:04Create talking avatars with InfiniteTalkThe demonstration shows how to sync a static character image with an audio file to create a talking avatar using the InfiniteTalk model and Wan 2.1.ComfyUIAI Avatar Video Generator
  6. 31:133:44Text-to-video with LTX-2The video walks through setting up the LTX-2 model in ComfyUI to generate high-resolution video clips from text prompts and images.ComfyUIAI Animation Generator
  7. 36:000:39Singing characters with LTX-2 and custom audioCurrentThe creator demonstrates how to use LTX-2 to make a character sing by providing a custom audio file and a specific singing prompt.ComfyUIAI Lip Sync Generator
  8. 37:281:08Upscale video with Seed-V2The user demonstrates upscaling a low-resolution AI-generated video to Full HD using the Seed-V2 workflow in ComfyUI for improved sharpness.ComfyUIAI Video Upscaler
  9. 39:075:27Cloud-based ComfyUI on RunPod/RunHubThe video shows how to run complex video workflows in the cloud using RunHub AI, demonstrating the interface and execution of InfiniteTalk and Wan 2.2 without local hardware.ComfyUIAI Animation Generator
  10. 44:340:52Frame interpolation for smoother motionThe user demonstrates a workflow to double the frame rate of a 16fps video to 32fps to create smoother motion in AI-generated clips.ComfyUIAI Video Interpolation
  11. Watch “ComfyUI Video Models: InfiniteTalk + Wan 2.2 + SCAIL + LTX-2 (Ep06)” →

AI Lip Sync Generator

  1. 14:523:11Lip-syncing audio to images with LTX 2.3The tutorial shows how to upload an MP3 file and a character image to generate a lip-synced video, including a demonstration of the voice isolation subgraph in ComfyUI.AIKnowledge2Go
  2. 36:000:39Singing characters with LTX-2 and custom audioCurrentThe creator demonstrates how to use LTX-2 to make a character sing by providing a custom audio file and a specific singing prompt.pixaroma
  3. 10:070:59Lip-syncing with custom audioThe creator demonstrates importing an audio file into the timeline and using a specific prompt structure to synchronize the character's mouth movements with the audio track.What Dreams Cost
  4. 5:411:07Configure frame window and motion settings for Infinite TalkThe user demonstrates how to adjust the frame window size and motion frame overlap to balance video smoothness, lip-sync consistency, and VRAM usage.pixaroma
  5. 8:411:11Create a multi-person talking video in ComfyUIThe creator demonstrates the multi-talk version of the workflow, showing how to assign separate audio files to two different characters in a single image using the multimodel setting.pixaroma
  6. 3:270:37Two-person image to video conversation in ComfyUIThe creator demonstrates how to use the 'add' setting in the ComfyUI node widget to load separate audio files for a back-and-forth conversation between two characters.ComfyUI Studio
  7. 4:041:22Parallel audio lip-sync for two charactersThe video shows how to use the 'PAR' setting in the WAN workflow to merge audio tracks for simultaneous or perfectly timed character interactions, including masking two faces in the node.ComfyUI Studio
  8. 5:420:22Video-to-video lip-sync with WAN workflowThe demonstration shows how to take an existing video and sync the lip movements to a new audio file using the ComfyUI workflow.ComfyUI Studio
  9. 4:460:35Installing and using the Sync custom node in ComfyUISebastian walks through installing the Sync custom node via Git URL in ComfyUI and setting up a workflow with video/audio input nodes and the Sync generate node.Sebastian Kamph
  10. 5:291:09Combining image-to-video models with Sync in ComfyUIThe video demonstrates a complex workflow using Krea.ai to transform a shot into a cinematic style before passing the generated video into Sync for final lip-syncing.Sebastian Kamph
  11. 3:201:14Animate static images with Infinite Talk V2A demonstration of the Infinite Talk V2 workflow where a static thumbnail image is paired with an audio track to create a lip-synced, animated talking character.Yaroflasher
  12. 4:341:29Video-to-video lip sync with Infinite TalkThe creator shows how to use a vid-to-vid workflow to analyze an audio file and automatically synchronize a character's mouth and expressions in an existing video clip.Yaroflasher