Allie 
Update: For those who are not programmers, you can run the latest Stable Diffusion directly from the website, https://huggingface.co/spaces/stabilityai/stable-diffusion.
Recently, I started playing around with Stable Diffusion. I enjoy generating art with Artificial Intelligence tools like Stable Diffusion. Stable Diffusion is a machine learning library trained on millions of images, photos, and artwork. The images are tagged with descriptive text that the model uses to take user prompts and generate new images. There is an example python notebook that uses Stable Diffusion from user generated prompts and converts them into images.
Stable Diffusion is a text-to-image latent diffusion model created by the researchers and engineers from CompVis, Stability AI and LAION. It’s trained on 512×512 images from a subset of the LAION-5B database. This model uses a frozen CLIP ViT-L/14 text encoder to condition the model on text prompts. With its 860M UNet and 123M text encoder, the model is relatively lightweight and runs on a GPU with at least 10GB VRAM.
Comment in Google Colabs example code.
Getting Started with Stable Diffusion
If you have a google account, then it is a simple matter to get set up with Google Colabs. Google Colabs runs python notebooks from your Google Drive. Once you enable Google Colabs, you can run the python example notebook created by Aayush Agrawal in the Google Colabs folder created on your Google Drive. Copy the python notebook to you Google Drives Colab folder. The actual notebook is located here on Google Colabs. Here is an article showing how to get an account on HuggingFace and run the code on Google Colabs.
Oops, The Libraries Have an Update
In the process of running the notebook, you may run into an error with an outdated import to the diffusers library. Comment out the first two lines and add in the third line to get the latest library.
#!pip install diffusers==0.4.0
#!pip install transformers scipy ftfy
!pip install --upgrade diffusers transformers scipy ftfy
!pip install "ipywidgets>=7,<8"Tips for Running Stable Diffusion
I have several tips to make running the python notebook easier:
- Keep your HuggingFace login token available in a text file to enter in when you run the code.
- I found that I wanted to save images on my own computer instead of overwriting the same file each time I ran one of the examples. I downloaded the created image to my computer when I wanted to save it.
- If you have a free account, you can only run 12 hours at a time. That’s really a lot if you exit and shutdown your runtime each time you want to quit. Otherwise, you might have to wait a couple of days before getting time again.
- Run the notebook with the GPUs turned on. This is much faster and the code is expecting it.
Some Example Prompts and Results
- “An outpost on an alien planet, desolate, poisonous atmosphere, highly detailed, vivid color, cinematic lighting, perfect composition, 8 k, gustave dore, derek zabrocki, greg rutkowski, belsinski, octane render”
- “Desolate arctic landscape, elegant, photorealistic, highly detailed, vivid color, cinematic lighting, perfect composition, 8 k, gustave dore, derek zabrocki, greg rutkowski, belsinski, octane render”
- “Landscape photography of the northern lights at night, elegant, highly detailed, subtle color, cinematic lighting, perfect composition”
I highly recommend playing around with Stable Diffusion and generating images. I had a lot of fun doing generating art with this tool.
