ComfyUI: Audio-to-Audio VAE Encoding
ComfyUIDemo summary
The demo shows loading an existing audio file (beats from Ace Step) into ComfyUI using VAE encoding to influence the style of the generated output.
Step-by-step
- Load the audio file into ComfyUI
- Apply a simple VAE encoding to the audio input
- Input an AI-generated prompt for the desired style
- Run the generation to remix the audio
Tips
- Use an AI-generated prompt to define specific genre characteristics like 'down tempo electronic' or 'heavy bass'
Highlights
“Probably not the very best way of doing it, but it's what I've got right now in comfy a simple VAE encoding.”
All demos from “Make High Quality Music in ComfyUI - Low VRAM!”
- 1:191:36Generate music with Stable Audio 3 Medium in ComfyUIThe creator demonstrates loading the Stable Audio 3 medium model in ComfyUI, setting up a prompt for 'gothic techno', and configuring the audio duration to 95 seconds before generating the track.ComfyUI· AI Music Generator
- 3:330:34Low VRAM audio generation with Small modelThe user switches to the 2GB 'small' model in ComfyUI to demonstrate audio generation suitable for low-end GPUs.ComfyUI· AI Music Generator
- 6:311:11AI Prompt Generation with GemmaThe creator shows an 'audio to text to audio' pipeline using Gemma to describe an audio input and generate a new prompt for the music generation node.ComfyUI· AI Music Generator
- 7:421:39Audio-to-Audio VAE EncodingCurrentThe demo shows loading an existing audio file (beats from Ace Step) into ComfyUI using VAE encoding to influence the style of the generated output.ComfyUI· AI Song Remixer
- 10:170:59Audio conditioning with voice inputThe user demonstrates using a 30-second vocal recording as an input combined with a text prompt and a linear quadratic scheduler to generate a new track.ComfyUI· AI Music Generator
- 11:161:07Generate sound effects with Stable AudioThe creator demonstrates generating specific sound effects like 'creaky doors' and 'underwater fireworks' using the medium and small sound effects models.ComfyUI· AI Sound Effect Generator
- 12:232:02Multi-sampler audio modificationThe video shows a workflow using two samplers in sequence, where the first stops at step four and the second continues, allowing for variations in the final audio output.ComfyUI· AI Audio Editor
- Watch “Make High Quality Music in ComfyUI - Low VRAM!” →
AI Song Remixer
- 7:421:39Audio-to-Audio VAE EncodingCurrentThe demo shows loading an existing audio file (beats from Ace Step) into ComfyUI using VAE encoding to influence the style of the generated output.Nerdy Rodent