demobook

Lip-syncing video with LipSync 2 Pro on sync.so

Demo summary

The user demonstrates uploading a video to sync.so, selecting the LipSync 2 Pro model, configuring settings like 'detect active speaker' and 'detect obstructions', and integrating ElevenLabs voices for the audio output.

Step-by-step

  1. Open sync.so and create a new project from the left sidebar
  2. Click the plus button and upload your video file
  3. Go to the model section and select LipSync 2 Pro
  4. Enable 'Detect active speaker' if the character is already moving their lips
  5. Enable 'Detect obstructions' if the face is partially blocked by hands or objects
  6. Upload an audio file or type a script manually in the input section
  7. Select a voice from the integrated ElevenLabs options
  8. Adjust Temperature, Stability, and Similarity Boost in the advanced settings
  9. Click Preview to check the audio before rendering
  10. Click Lip Sync to process and download the final clip

Options

  • Use React 1 model instead of LipSync 2 Pro for rewriting entire facial performances
  • Upload an existing audio file or use text-to-speech
  • Adjust Temperature (0.0 to 1.0) to control expressiveness
  • Adjust Stability (0.0 to 1.0) to balance consistency and performance

Watch out for

  • The character's head should not be moving excessively for obstruction detection to work
  • If you forget to enable 'Detect active speaker' on talking footage, the model may misalign the mouth region
  • Avoid setting Temperature to 1.0 as it causes unnatural variations and instability

Tips

  • Use silent footage for the cleanest results as the tool can build mouth movements from scratch
  • Set Temperature to 0.9 as the 'sweet spot' for expressiveness without instability
  • Set Stability to 0.5 to keep the performance natural rather than robotic
  • Set Similarity Boost to 0.8 or 0.9 to keep the voice close to the ElevenLabs original without sounding unnatural
  • Always use the Preview function first to avoid wasting credits on a full video render

Highlights

Understanding which model to use is the single most important decision in this workflow.

All demos from “How to Make Any Video Talk Naturally With AI Lip Sync

  1. 1:433:02Lip-syncing video with LipSync 2 Pro on sync.soCurrentThe user demonstrates uploading a video to sync.so, selecting the LipSync 2 Pro model, configuring settings like 'detect active speaker' and 'detect obstructions', and integrating ElevenLabs voices for the audio output.SSync.SoAI Lip Sync Generator
  2. 6:551:34Changing facial expressions with React 1 on sync.soThe demonstration shows how to use the React 1 model on sync.so to rewrite facial emotions (such as happy, sad, or angry) and head movements while syncing the mouth to new dialogue.SSync.SoAI Lip Sync Generator
  3. Watch “How to Make Any Video Talk Naturally With AI Lip Sync” →

AI Lip Sync Generator

  1. 1:433:02Lip-syncing video with LipSync 2 Pro on sync.soCurrentThe user demonstrates uploading a video to sync.so, selecting the LipSync 2 Pro model, configuring settings like 'detect active speaker' and 'detect obstructions', and integrating ElevenLabs voices for the audio output.AI Simplified
  2. 6:551:34Changing facial expressions with React 1 on sync.soThe demonstration shows how to use the React 1 model on sync.so to rewrite facial emotions (such as happy, sad, or angry) and head movements while syncing the mouth to new dialogue.AI Simplified
  3. 1:220:39Translate and lip-sync video with Sync.soThe creator demonstrates the Sync.so web interface by creating a new project, uploading a video, and pasting translated Spanish text to generate a lip-synced output.Sebastian Kamph
  4. 2:111:30Advanced voice cloning and obstruction detection in Sync.soThe demo shows how to upload an audio sample for voice cloning and toggle the 'detect obstructions' setting to maintain lip-sync accuracy when a hand covers the mouth.Sebastian Kamph
  5. 0:451:53Lip sync video with Sync.soThe creator demonstrates how to upload a video and audio file to the Sync.so dashboard and apply the AI lip-syncing process to align the character's mouth movements with the audio.Learn with Emnet
  6. 2:341:43Navigating the Sync.so Dashboard and Upload ProcessThe video demonstrates the Sync.so interface, showing how to toggle between Light and Timeline modes and how to upload video and audio files for AI lip-syncing.IdeaPlex