Higgsfield: Final video generation with audio and video references

Higgsfield Try it →Watch full video →MMira AI

Demo summary

The user performs a complex task in Higgsfield by combining a timeline prompt, a character board, and a blacked-out video reference to sync audio and visuals for a final dance clip.

Step-by-step

Write a detailed timeline prompt specifying actions for specific time intervals
Use an LLM like Claude to polish the timeline prompt
Upload a character board to use as a locked reference
Upload a video reference that contains the desired audio track
Cover the video reference with a solid black screen to isolate the audio
Tag the references within the prompt to define their specific roles
Click generate

Watch out for

SeaDance cannot take audio files on their own yet; audio must be embedded in a video file
You must black out the video reference to prevent the AI from being confused by old visuals or 'bleeding' them into the new generation

Tips

Use a blacked-out video reference to ensure the AI only takes the sound from a previous clip
Tag your references in the prompt so the AI knows exactly what each one is for

Highlights

“here's what makes this work on the first try”

All demos from “How To Make AI Videos - Prompting Guide For Beginners”

1:010:20Navigating the Higgsfield UIThe narrator walks through the Higgsfield interface, demonstrating how to log in, toggle to the video section, select the Seedance 2.0 model, and use the prompt box and reference image upload area.Higgsfield· AI Video Generator
4:020:12Applying a structured prompt formulaThe video shows the process of attaching a character board and using a specific 'action-setting-camera-style' formula to generate a tracking shot of a character in New York.LLuma Dream Machine· AI Video Prompt Generator
7:070:25Timeline prompting for controlled motionThe narrator demonstrates 'timeline prompting' by breaking a 15-second clip into specific time chunks (0-5s, 5-10s, 10-15s) to ensure deliberate and calm character movements.LLuma Dream Machine· AI Animation Generator
8:241:08Final video generation with audio and video referencesCurrentThe user performs a complex task in Higgsfield by combining a timeline prompt, a character board, and a blacked-out video reference to sync audio and visuals for a final dance clip.Higgsfield· AI Animation Generator
Watch “How To Make AI Videos - Prompting Guide For Beginners” →

AI Animation Generator

12:311:40AI Video Character Replacement WorkflowA demonstration of a complex workflow where a generated character is merged into a separate scene asset to create an animated video output.DraftID
6:471:30Directing video scenes with emotion controlsConor demonstrates the Higgsfield video mode interface, selecting saved characters and locations while setting specific emotional states like 'serenity' and 'joy' to influence the generated clip.Creating with Conor
1:041:40Create a consistent AI character in Higgsfield Cinema StudioThe user demonstrates how to use the 'Cast' feature in Higgsfield to build a character from scratch by selecting genre, budget, era, archetype, and physical traits like hair and eye color.Get365AI
3:491:51Direct a multi-shot AI video sequenceThe user shows how to combine a saved character and environment in the Director panel, set camera movements, and use the 'multi-shot auto' feature to generate a consistent cinematic scene.Get365AI
9:371:15Configure video generation settings and character emotionsThe narrator walks through the director panel in Higgsfield, dropping in character and location assets, and using the 'smile icon' to set specific character emotions like 'vigilance' for the scene.Youri van Hofwegen
8:241:08Final video generation with audio and video referencesCurrentThe user performs a complex task in Higgsfield by combining a timeline prompt, a character board, and a blacked-out video reference to sync audio and visuals for a final dance clip.Mira AI
3:381:17Setting up a scene in Sea Dance 2The creator navigates the Higgsfield UI to access Sea Dance 2, uploads a key visual image reference, and adds a character sheet to maintain consistency.Jack Vs. AI
7:182:10Animate talking characters with Higgsfield Kling 3.0The creator shows how to use the Kling 3.0 model in Higgsfield, uploading a reference image and providing a text-to-video prompt with specific dialogue in quotation marks to animate the characters.Taro Creates
1:110:06Apply Dolly In preset in Cinema StudioThe user opens the movements panel in Higgsfield, selects the 'dolly in' preset, and enters a scene description to generate a camera move.Youri van Hofwegen
2:460:04Dolly Left/Right lateral slideThe user selects the lateral movement preset and pastes a prompt to demonstrate parallax depth in the AI-generated scene.Youri van Hofwegen
4:050:08Custom 180-degree orbitThe user pastes a custom prompt to control the exact start and stop points of a camera orbit around a character.Youri van Hofwegen
10:070:13Leading tracking shot presetThe creator selects the 'camera follows' preset and enters a prompt to have the camera pull back as the subject walks forward.Youri van Hofwegen