demobook

ComfyUI: Guiding Structure with ControlNet

ComfyUITry it →Watch full video →pixaromapixaroma·

Demo summary

The video shows how to use Canny and Depth ControlNet models with pre-processors to guide image generation based on an input sketch or 3D text.

Step-by-step

  1. Download the Canny, Depth, and OpenPose ControlNet models
  2. Place the downloaded models into the 'models/control_net' folder within your ComfyUI directory
  3. Press the 'R' key to refresh node definitions if ComfyUI was already open
  4. Add an 'Apply ControlNet' node and connect the positive and negative conditioning
  5. Add a 'Load ControlNet Model' node and connect it to the 'Apply ControlNet' node
  6. Add a 'Load Image' node and connect it to an 'AIO' (All-In-One) pre-processor node
  7. Select the 'Canny Edge' pre-processor to match the Canny ControlNet model
  8. Connect the pre-processor output to the 'Apply ControlNet' node and run the workflow

Options

  • Use Canny, Depth, or OpenPose models depending on the desired guidance
  • Adjust the resolution in the pre-processor node for more detail in the map

Watch out for

  • ControlNet models are only compatible with the specific base model they were trained for
  • You must install the specific custom nodes required for this workflow if not using the 'easy install' version of ComfyUI
  • Do not connect a raw image directly to ControlNet; it requires a pre-processor to convert the image into a format it understands

Tips

  • Organize your ControlNet models into subfolders named after their compatible base models
  • Color-code your ControlNet-related nodes (e.g., yellow) to make the workflow easier to read
  • Add a 'Preview Image' node after the pre-processor to see exactly what guidance map is being sent to the model

Highlights

“without control net, it would be very hard to get something this complex using only a prompt”

All demos from “ComfyUI Course - Learn ComfyUI From Scratch | Full 5 Hour Course (Ep01)”

  1. 9:173:04Install ComfyUI using Easy InstallThe instructor demonstrates how to download the ComfyUI Easy Install zip from GitHub, extract it, and run the BAT file to set up a portable local installation.ComfyUIAI Image Generator
  2. 13:355:23ComfyUI Interface OverviewA walkthrough of the ComfyUI UI, demonstrating the canvas, node search, top bar controls, view settings, and how to properly close the application.ComfyUIAI Image Generator
  3. 18:588:14Running a Ready-Made WorkflowThe video shows how to load a .json workflow, download the required Juggernaut Reborn model, refresh the node list, and generate an image of a cat.ComfyUIAI Image Generator
  4. 27:266:13Building a Basic Image Processing WorkflowDemonstration of adding nodes (Load Image, Image Crop, Save Image) and connecting them via color-coded links to perform non-AI image tasks.ComfyUIAI Crop Image
  5. 44:186:05Using Math and Grouping NodesThe instructor shows how to use Math Int and Preview as Text nodes for automation and how to organize nodes into Groups for better workflow management.ComfyUIAI Image Generator
  6. 1:27:395:06Image-to-Image Workflow with Denoise ControlDemonstration of converting a text-to-image workflow into an image-to-image setup using VAE Encode and adjusting the denoise parameter to control variation.ComfyUIImage to Image
  7. 1:48:162:58Managing Custom Nodes with ComfyUI ManagerThe video walks through searching for, installing, and uninstalling custom nodes (specifically the 'Align' node) using the built-in Manager.ComfyUIAI Image Generator
  8. 2:35:539:48Applying LoRA for Style ModificationDemonstration of adding a LoRA loader to a workflow, using trigger words, and adjusting LoRA strength to change a robot into a 'cake style' robot.ComfyUIAI Style Transfer
  9. 2:47:359:01Guiding Structure with ControlNetCurrentThe video shows how to use Canny and Depth ControlNet models with pre-processors to guide image generation based on an input sketch or 3D text.ComfyUIAI Image Generator
  10. 3:11:488:27Generating Images with Z-Image TurboA demonstration of setting up the modern Z-Image Turbo model, including the specific Model Sampling AuraFlow node and high-quality prompt generation.ComfyUIAI Image Generator
  11. 4:13:595:52Integrating ChatGPT API for Prompt GenerationThe instructor demonstrates signing into the ComfyUI API and using a ChatGPT node to automatically generate detailed image prompts for local models.ComfyUIAI Image Generator
  12. Watch “ComfyUI Course - Learn ComfyUI From Scratch | Full 5 Hour Course (Ep01)” →

AI Image Generator

  1. 1:560:39Load a text-to-image workflow and download missing modelsThe user demonstrates how to load a default text-to-image template in ComfyUI and use the 'show missing models' feature to download required AI checkpoints.Kevin Stratvert
  2. 4:110:29Manage generated assets in ComfyUIThe presenter demonstrates how to access the assets panel using the 'A' shortcut to view, save, or delete previously generated images.Kevin Stratvert
  3. 15:005:45Generate image with Hermes Agent and ComfyUIThe host uses the Hermes Agent TUI to request a 1024x1024 image of a tree using the Zimage Turbo model, which the agent executes by generating and running a ComfyUI workflow.ComfyUI
  4. 6:420:59Configure Gemma 2B Text Encoder in ComfyUIThe creator shows how to load the Gemma 2B text encoder model into the ComfyUI workflow and select it from the node dropdown menu.AI Search
  5. 4:270:30Copying and replacing prompts in ComfyUIThe user demonstrates a custom node feature that allows hovering over prompt examples to copy and then replace the current prompt in the workflow with a single click.pixaroma
  6. 13:030:51Managing node colors with PixaRoma nodesThe video demonstrates UI updates for PixaRoma nodes in ComfyUI, including selecting color swatches, copying/pasting colors between nodes, and setting favorite colors.pixaroma
  7. 4:150:35Launch ComfyUI on WindowsThe creator demonstrates how to launch the portable version of ComfyUI by running the 'run_nvidia_gpu' batch file and accessing the interface via a web browser.WINBUSH
  8. 10:071:30Browse and load ComfyUI templatesThe creator demonstrates how to navigate the templates menu, filter for local models, and load a Flux 2 image generation workflow.WINBUSH
  9. 0:000:33Configuring MSR mode in LTX DirectorThe user demonstrates setting up the Director node in ComfyUI for MSR (Multiple Subject Reference) mode, including bypassing clean latent slices and enabling the MSR LoRA.RedShade
  10. 5:090:19Loading LTX Director V5 workflow templateThe video shows how to search for and load the LTX Director V5 template within ComfyUI and input prompts for character generation.RedShade
  11. 4:251:21Iterative rerolling at specific generation stagesThe video shows how to use the 'reroll' button at specific stages of the workflow to change motion or visual quality without re-generating the initial seed samples.Fox•Fur•Essence Films
  12. 9:173:04Install ComfyUI using Easy InstallThe instructor demonstrates how to download the ComfyUI Easy Install zip from GitHub, extract it, and run the BAT file to set up a portable local installation.pixaroma
demobook

Show, don’t tell.

Categories

LaunchesEmail MarketingImage Generation & EditingMusic & AudioSales TechVideo & Animation

Trends

AI Fruit Video

About

SubmitAboutPrivacyTermsContact

© demobook.co