Automatic1111 and ControlNet

"AUTOMATIC1111/stable-diffusion-webui" is dedicated to a web-based user interface for the Stable Diffusion model. This UI facilitates the generation and manipulation of images using Stable Diffusion, a neural network model specializing in creating images from text descriptions. The UI includes features like text-to-image (txt2img) and image-to-image (img2img) transformations, catering to a broad range of AI-driven image generation and editing tasks. The repository contains code predominantly in Python and JavaScript, and is licensed under AGPL-3.0*.

For more detailed information, you can explore the repository directly on GitHub: https://github.com/AUTOMATIC1111/stable-diffusion-webui.

Installation Guide: 

https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/README.md

Installation and Running

• Make sure the required dependencies are met and follow the instructions available for:
• NVidia (useful step-by-step video available)
• AMD GPUs.
• Intel CPUs, Intel GPUs (both integrated and discrete) (external wiki page)

Alternatively, use online services (like Google Colab):

• List of Online Services

Installation on Windows 10/11 with NVidia-GPUs using release package

• Download sd.webui.zip from v1.0.0-pre and extract its contents.
• Run update.bat.
• Run run.bat.
• For more details see Install-and-Run-on-NVidia-GPUs

Automatic Installation on Windows

• Install Python 3.10.6 (Newer version of Python does not support torch), checking "Add Python to PATH".
• Install git.
• Download the stable-diffusion-webui repository, for example by running git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git.
• Run webui-user.bat from Windows Explorer as normal, non-administrator, user.

ControlNet

ControlNet is a neural network that enhances text-to-image models like Stable Diffusion by adding precise control over image elements such as shapes, depth, and human poses, making image generation more versatile and accurate. 

This is very useful for quick architectural visualization. For example, based on provided sketch outlines, users can experiment with various moods and contextual settings. They can also transfer specific elements from one image to another, creating multiple iterations according to their prompts.

There’s a standalone Github repository for ControlNet (https://github.com/lllyasviel/ControlNet), maintained by user Illyasviel, and an Extension for popular SD interface Automatic1111 (https://github.com/Mikubill/sd-webui-controlnet), maintained by user Mikubill.

ControlNet Installation Guide: 

https://github.com/Mikubill/sd-webui-controlnet/blob/main/README.md

ControlNet is an extension for AUTOMATIC1111's Stable Diffusion web UI. It allows the Web UI to add ControlNet to the original Stable Diffusion model to generate images. This new addition will be incorporated without extra steps to combine them.

Installation 

• Open "Extensions" tab.
• Open "Install from URL" tab in the tab.
• Enter https://github.com/Mikubill/sd-webui-controlnet.git to "URL for extension's git repository".
• Press "Install" button.
• Wait for 5 seconds, and you will see the message "Installed into stable-diffusion-webui\extensions\sd-webui-controlnet. Use Installed tab to restart".
• Go to "Installed" tab, click "Check for updates", and then click "Apply and restart UI". (The next time you can also use these buttons to update ControlNet.)
• Completely restart A1111 webui including your terminal. (If you do not know what is a "terminal", you can reboot your computer to achieve the same effect.)
• Download models (see below).
• After you put models in the correct folder, you may need to refresh to see the models. The refresh button is right to your "Model" dropdown.

*AGPL-3.0 stands for the Affero General Public License version 3.0. It's a free, copyleft license for software and other kinds of works, ensuring that all modified versions of the licensed software are also free and open source. Specifically, it requires that the source code be made available to any network user of the AGPL-licensed work, typically used in web applications. This means if you modify an AGPL-licensed program and run it on a server accessible to others, you must also share the modified source code under AGPL.

Related Links and References:

• https://www.youtube.com/watch?v=hr2ri5SBPfU (install ControlNet)
• https://education.civitai.com/civitai-guide-to-controlnet/
• https://arxiv.org/abs/2302.05543
• https://education.civitai.com/civitai-guide-to-controlnet/#what-is-controlnet

Stable Diffusion Extra Large (SDXL) Model download from Civitai for ControlNet:

• Kohya-ss Canny, Edge detection model for highlighting structural details - https://civitai.com/models/136070?modelVersionId=158416
• Depth, Kohya-ss, Depth analysis for spatial and 3D effects in images  - https://civitai.com/models/136070?modelVersionId=155265
• Lineart, TencentARC t2i Adapter, Converts images into a line-art style, suitable for sketches and architectural drawings - https://civitai.com/models/136070?modelVersionId=155414
• SD Control Collection - https://huggingface.co/lllyasviel/sd_control_collection/tree/main

After downloading a ControlNet model, save it in the stable-diffusion-webui\extensions\sd-webui-controlnet\models\controlnet directory for Automatic1111*, or in the ComfyUI\models\controlnet directory for ComfyUI**. Restart the interface afterward to ensure the model is recognized.

• LoRA - https://civitai.com/search/models?sortBy=models_v5&query=LoRA

*Automatic1111 is user-friendly and straightforward, ideal for casual users or beginners who want to create images with Stable Diffusion without needing deep technical knowledge. 

**ComfyUI, with its node-based interface, offers more detailed control and customization, suited for advanced users or professionals who require precise control over image generation processes. ComfyUI has a steeper learning curve but provides greater flexibility for complex tasks.

Souce: 

• https://education.civitai.com/civitai-guide-to-controlnet/
• https://stable-diffusion-art.com/install-windows/#What_Python_version_should_I_use

Without downloading of models for explorations: https://www.segmind.com/models