Higgsfield: Final video generation with audio and video references

Higgsfield Try it →Watch full video →MMira AI

Demo summary

The user performs a complex task in Higgsfield by combining a timeline prompt, a character board, and a blacked-out video reference to sync audio and visuals for a final dance clip.

Step-by-step

Write a detailed timeline prompt specifying actions for specific time intervals
Use an LLM like Claude to polish the timeline prompt
Upload a character board to use as a locked reference
Upload a video reference that contains the desired audio track
Cover the video reference with a solid black screen to isolate the audio
Tag the references within the prompt to define their specific roles
Click generate

Watch out for

SeaDance cannot take audio files on their own yet; audio must be embedded in a video file
You must black out the video reference to prevent the AI from being confused by old visuals or 'bleeding' them into the new generation

Tips

Use a blacked-out video reference to ensure the AI only takes the sound from a previous clip
Tag your references in the prompt so the AI knows exactly what each one is for

Highlights

“here's what makes this work on the first try”

All demos from “How To Make AI Videos - Prompting Guide For Beginners”

1:010:20Navigating the Higgsfield UIThe narrator walks through the Higgsfield interface, demonstrating how to log in, toggle to the video section, select the Seedance 2.0 model, and use the prompt box and reference image upload area.Higgsfield· AI Video Generator
4:020:12Applying a structured prompt formulaThe video shows the process of attaching a character board and using a specific 'action-setting-camera-style' formula to generate a tracking shot of a character in New York.LLuma Dream Machine· AI Video Prompt Generator
7:070:25Timeline prompting for controlled motionThe narrator demonstrates 'timeline prompting' by breaking a 15-second clip into specific time chunks (0-5s, 5-10s, 10-15s) to ensure deliberate and calm character movements.LLuma Dream Machine· AI Animation Generator
8:241:08Final video generation with audio and video referencesCurrentThe user performs a complex task in Higgsfield by combining a timeline prompt, a character board, and a blacked-out video reference to sync audio and visuals for a final dance clip.Higgsfield· AI Animation Generator
Watch “How To Make AI Videos - Prompting Guide For Beginners” →

AI Animation Generator

12:311:40AI Video Character Replacement WorkflowA demonstration of a complex workflow where a generated character is merged into a separate scene asset to create an animated video output.DraftID
1:041:40Create a consistent AI character in Higgsfield Cinema StudioThe user demonstrates how to use the 'Cast' feature in Higgsfield to build a character from scratch by selecting genre, budget, era, archetype, and physical traits like hair and eye color.Get365AI
3:491:51Direct a multi-shot AI video sequenceThe user shows how to combine a saved character and environment in the Director panel, set camera movements, and use the 'multi-shot auto' feature to generate a consistent cinematic scene.Get365AI
6:471:30Directing video scenes with emotion controlsConor demonstrates the Higgsfield video mode interface, selecting saved characters and locations while setting specific emotional states like 'serenity' and 'joy' to influence the generated clip.Creating with Conor
7:182:10Animate talking characters with Higgsfield Kling 3.0The creator shows how to use the Kling 3.0 model in Higgsfield, uploading a reference image and providing a text-to-video prompt with specific dialogue in quotation marks to animate the characters.Taro Creates
9:371:15Configure video generation settings and character emotionsThe narrator walks through the director panel in Higgsfield, dropping in character and location assets, and using the 'smile icon' to set specific character emotions like 'vigilance' for the scene.Youri van Hofwegen
3:511:27Directing multi-shot sequences in Manual ModeThe user walks through the Manual Mode interface in Higgsfield, setting scene durations, camera movements (tracking, orbit, dolly), and character emotion/genre tags for a multi-shot video.Higgsfield AI
5:250:51Generating a horror sequence with character referencesThe user demonstrates uploading character reference sheets and a keyframe prompt to generate a consistent multi-shot horror sequence in Higgsfield Cinema Studio 2.Higgsfield AI
8:241:08Final video generation with audio and video referencesCurrentThe user performs a complex task in Higgsfield by combining a timeline prompt, a character board, and a blacked-out video reference to sync audio and visuals for a final dance clip.Mira AI
3:381:17Setting up a scene in Sea Dance 2The creator navigates the Higgsfield UI to access Sea Dance 2, uploads a key visual image reference, and adds a character sheet to maintain consistency.Jack Vs. AI
2:571:42Navigating Kling 3.0 settings in HiggsfieldThe creator demonstrates how to access Kling 3.0 models within the Higgsfield interface and explains settings for duration, aspect ratio, and native audio.Sebastien Jefferies
7:252:10Setting up character Elements for consistencyThe user shows how to create 'Elements' by uploading character reference images and providing descriptions to maintain subject consistency across AI video generations.Sebastien Jefferies