Higgsfield: Final video generation with audio and video references

Higgsfield Try it →Watch full video →MMira AI

Demo summary

The user performs a complex task in Higgsfield by combining a timeline prompt, a character board, and a blacked-out video reference to sync audio and visuals for a final dance clip.

Step-by-step

Write a detailed timeline prompt specifying actions for specific time intervals
Use an LLM like Claude to polish the timeline prompt
Upload a character board to use as a locked reference
Upload a video reference that contains the desired audio track
Cover the video reference with a solid black screen to isolate the audio
Tag the references within the prompt to define their specific roles
Click generate

Watch out for

SeaDance cannot take audio files on their own yet; audio must be embedded in a video file
You must black out the video reference to prevent the AI from being confused by old visuals or 'bleeding' them into the new generation

Tips

Use a blacked-out video reference to ensure the AI only takes the sound from a previous clip
Tag your references in the prompt so the AI knows exactly what each one is for

Highlights

“here's what makes this work on the first try”

All demos from “How To Make AI Videos - Prompting Guide For Beginners”

1:010:20Navigating the Higgsfield UIThe narrator walks through the Higgsfield interface, demonstrating how to log in, toggle to the video section, select the Seedance 2.0 model, and use the prompt box and reference image upload area.Higgsfield· AI Video Generator
4:020:12Applying a structured prompt formulaThe video shows the process of attaching a character board and using a specific 'action-setting-camera-style' formula to generate a tracking shot of a character in New York.LLuma Dream Machine· AI Video Prompt Generator
7:070:25Timeline prompting for controlled motionThe narrator demonstrates 'timeline prompting' by breaking a 15-second clip into specific time chunks (0-5s, 5-10s, 10-15s) to ensure deliberate and calm character movements.LLuma Dream Machine· AI Animation Generator
8:241:08Final video generation with audio and video referencesCurrentThe user performs a complex task in Higgsfield by combining a timeline prompt, a character board, and a blacked-out video reference to sync audio and visuals for a final dance clip.Higgsfield· AI Animation Generator
Watch “How To Make AI Videos - Prompting Guide For Beginners” →

AI Animation Generator

12:311:40AI Video Character Replacement WorkflowA demonstration of a complex workflow where a generated character is merged into a separate scene asset to create an animated video output.DraftID
1:041:40Create a consistent AI character in Higgsfield Cinema StudioThe user demonstrates how to use the 'Cast' feature in Higgsfield to build a character from scratch by selecting genre, budget, era, archetype, and physical traits like hair and eye color.Get365AI
3:491:51Direct a multi-shot AI video sequenceThe user shows how to combine a saved character and environment in the Director panel, set camera movements, and use the 'multi-shot auto' feature to generate a consistent cinematic scene.Get365AI
7:182:10Animate talking characters with Higgsfield Kling 3.0The creator shows how to use the Kling 3.0 model in Higgsfield, uploading a reference image and providing a text-to-video prompt with specific dialogue in quotation marks to animate the characters.Taro Creates
3:511:27Directing multi-shot sequences in Manual ModeThe user walks through the Manual Mode interface in Higgsfield, setting scene durations, camera movements (tracking, orbit, dolly), and character emotion/genre tags for a multi-shot video.Higgsfield AI
5:250:51Generating a horror sequence with character referencesThe user demonstrates uploading character reference sheets and a keyframe prompt to generate a consistent multi-shot horror sequence in Higgsfield Cinema Studio 2.Higgsfield AI
2:571:42Navigating Kling 3.0 settings in HiggsfieldThe creator demonstrates how to access Kling 3.0 models within the Higgsfield interface and explains settings for duration, aspect ratio, and native audio.Sebastien Jefferies
7:252:10Setting up character Elements for consistencyThe user shows how to create 'Elements' by uploading character reference images and providing descriptions to maintain subject consistency across AI video generations.Sebastien Jefferies
4:140:49Configure video generation in Higgsfield Seedance 2.0The user demonstrates how to select the 'Enhanced 2.0 Fast Unlimited' model, set the video duration to 15 seconds, and adjust the resolution to 720p within the Higgsfield interface.JSFILMZ
8:000:37Adjusting video bitrate in HiggsfieldThe creator shows the bitrate slider in the Higgsfield UI, setting it to 'High' for less compression before initiating a generation.JSFILMZ
13:160:39Manage project elements in Higgsfield CollabThe user shows how to add character sheets, locations, and props as 'Elements' in the Higgsfield interface so they auto-attach when pasting prompts.Higgsfield AI
47:031:53Build AI workflows in Higgsfield CanvasA demonstration of the node-based Canvas feature, connecting a prompt node to an image generation node and then to a Kling 3.0 video node to create an animation.Artturi Jalli