How to Deploy Stable Diffusion (Automatic1111)

A guide to deploying a custom stable diffusion model on SaladCloud with Automatic1111

Last Updated: October 10, 2024

High Level

Regardless of your choice of stable diffusion inference server, models, or extensions, the basic process is as follows:

1. Get a docker image that runs your inference server
2. Copy any models and extensions you want into the docker image
3. Ensure the container is listening on an ipv6 address
4. Push the new image up to a container registry
5. Deploy the image as a SaladCloud container group

Find a Docker Image

Find a docker image of Automatic1111. Here’s one we’ve verified works on Salad:

Automatic1111

• Git Repo: https://github.com/SaladTechnologies/stable-diffusion-webui-docker
• Docker Image: saladtechnologies/a1111:ipv6-latest
• Data Directory: /data
• Model Directory: /data/models
• Extension Directory: /data/config/auto/extensions
• Controlnet Model Directory (If controlnet extension is installed): /data/config/auto/extensions/sd-webui-controlnet/models

Note that you will be interacting with this as an API, and not through the browser user interface.

Download Your Models and Extensions

Download any model files you plan to use. For our example, we’re going to use Dreamshaper 8 , available on Civitai.com (https://blog.salad.com/civitai-salad/)

Create a Dockerfile

1. Create a new file called Dockerfile and open it in your preferred text editor. At this point, your directory should look like this:

   .
   ├── Dockerfile
   └── dreamshaper_8.safetensors
   

2. Copy the following into your Dockerfile:

# We're going to use this verified Automatic1111 image as a base
# It's best to use a pinned version to protect you from unexpected API
# changes in the future
FROM saladtechnologies/a1111:ipv6-v1.9.4

# Now we copy our model into the image
ENV MODEL_DIR=/data/models
COPY dreamshaper_8.safetensors ${MODEL_DIR}/Stable-diffusion/dreamshaper_8.safetensors

# We set A1111 to listen on the ipv4 port 7859, and then use socat to forward traffic
# from the ipv6 port 7860 (or whatever you set PORT to) to the ipv4 port 7859
ENV PORT=7860
CMD python -u webui.py --listen --port 7859 --xformers --api \
  & socat TCP6-LISTEN:${PORT},fork TCP4:127.0.0.1:7859;

Build and Test Your Docker Image

1. Build the docker image. You should change the specified tag to suit your purpose.

docker build -t saladtechnologies/a1111:dreamshaper-8 .

2. (Recommended) Run the docker image locally to confirm it works as expected

docker run -it --rm --gpus all -p 7860:7860 --name a1111 \
saladtechnologies/a1111:dreamshaper-8

Navigate to http://localhost:7860/docs in your browser to see the API docs for Automatic1111.

3. Test a Text-to-Image request

See the docs for more information on submitting a text-to-image request. Here’s an example JSON request body:

{
  "prompt": "A beautiful sunset over a calm lake"
}

Submit this to the /sdapi/v1/txt2img endpoint as a POST request:

curl -X 'POST' \
  'http://localhost:7860/sdapi/v1/txt2img' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "prompt": "A beautiful sunset over a calm lake"
}'

You will receive back a response in JSON format, including the generated image in Base64 encoded format:

{
  "images": ["base64encodedimage"],
  "parameters": {
    "prompt": "A beautiful sunset over a calm lake",
    "negative_prompt": "",
    "styles": null,
    "seed": -1,
    "subseed": -1,
    "subseed_strength": 0,
    "seed_resize_from_h": -1,
    "seed_resize_from_w": -1,
    "sampler_name": null,
    "batch_size": 1,
    "n_iter": 1,
    "steps": 50,
    "cfg_scale": 7,
    "width": 512,
    "height": 512,
    "restore_faces": null,
    "tiling": null,
    "do_not_save_samples": false,
    "do_not_save_grid": false,
    "eta": null,
    "denoising_strength": null,
    "s_min_uncond": null,
    "s_churn": null,
    "s_tmax": null,
    "s_tmin": null,
    "s_noise": null,
    "override_settings": null,
    "override_settings_restore_afterwards": true,
    "refiner_checkpoint": null,
    "refiner_switch_at": null,
    "disable_extra_networks": false,
    "comments": null,
    "enable_hr": false,
    "firstphase_width": 0,
    "firstphase_height": 0,
    "hr_scale": 2,
    "hr_upscaler": null,
    "hr_second_pass_steps": 0,
    "hr_resize_x": 0,
    "hr_resize_y": 0,
    "hr_checkpoint_name": null,
    "hr_sampler_name": null,
    "hr_prompt": "",
    "hr_negative_prompt": "",
    "sampler_index": "Euler",
    "script_name": null,
    "script_args": [],
    "send_images": true,
    "save_images": false,
    "alwayson_scripts": {}
  },
  "info": "{\"prompt\": \"A beautiful sunset over a calm lake\", \"all_prompts\": [\"A beautiful sunset over a calm lake\"], \"negative_prompt\": \"\", \"all_negative_prompts\": [\"\"], \"seed\": 726538242, \"all_seeds\": [726538242], \"subseed\": 4102306259, \"all_subseeds\": [4102306259], \"subseed_strength\": 0, \"width\": 512, \"height\": 512, \"sampler_name\": \"Euler\", \"cfg_scale\": 7.0, \"steps\": 50, \"batch_size\": 1, \"restore_faces\": false, \"face_restoration_model\": null, \"sd_model_name\": \"dreamshaper_8\", \"sd_model_hash\": \"879db523c3\", \"sd_vae_name\": null, \"sd_vae_hash\": null, \"seed_resize_from_w\": -1, \"seed_resize_from_h\": -1, \"denoising_strength\": null, \"extra_generation_params\": {}, \"index_of_first_image\": 0, \"infotexts\": [\"A beautiful sunset over a calm lake\\nSteps: 50, Sampler: Euler, CFG scale: 7.0, Seed: 726538242, Size: 512x512, Model hash: 879db523c3, Model: dreamshaper_8, Version: v1.7.0\"], \"styles\": [], \"job_timestamp\": \"20240731152922\", \"clip_skip\": 1, \"is_using_inpainting_conditioning\": false, \"version\": \"v1.7.0\"}"
}

4. Decode the base64 encoded string into your image. You can do this in a free browser tool such as https://codebeautify.org/base64-to-image-converter

or using CLI tools like jq and base64. For this method, first save your response to a file called response.json. Then, run the following command:

jq -r '.images[0]' response.json | base64 -d > image.png

Push and Deploy Your Docker Image

1. Push your docker image up to docker hub (or the container registry of your choice.)

docker push saladtechnologies/a1111:dreamshaper-8

2. Deploy your image on Salad, using either the Portal or the SaladCloud Public API

   

   We’re going to name our container group something obvious, and fill in the configuration form. We’re going to use 3 replicas, to ensure coverage during node interruptions and reallocations.

   

   Since this is a stable diffusion 1.5 based model, we’re gong to give ourselves fairly modest hardware: 4 vCPUs, 12GB ram, and an RTX 3060 Ti GPU.

   

   We want to add the container gateway to our deployment, so that we will get a URL we can use to access it. Make sure to set the port to 7860, or whatever you set the PORT environment variable to in your Dockerfile.

   

   We need to enable a startup probe and a liveness probe, to make sure the container gateway only routes requests to nodes that are ready for them.

   

Interact with Your Deployment

1. Wait for the deployment to be ready.

   1. First, SaladCloud pulls your container image into our own internal high-performance cache.

      

   2. Next, SaladCloud locates eligible nodes for your workload, based on the configuration you provided.

      

   3. Next, SaladCloud begins downloading the cached container image to the nodes that have been assigned to your workload.

      

      This step can take tens of minutes in some cases, depending on the size of the image, and the internet speed of the individual nodes. Note that our progress bars are only estimates, and do not necessarily reflect real-time download status. These slow cold starts, and the possibility of nodes being interrupted by their host without warning, are why we always want to provision multiple replicas.

   4. Eventually, you will see instances listed as “running”, with a green check in the “ready” column.

      

2. Submit your prompt to the provided Access Domain Name. You will get back a json response within a few seconds. See above for how to submit the request and process the response.