An SDXL Random Artist Collection — Meta Data Lost and Lesson Learned

Stable Diffusion SDXL Wildcards and ComfyUI

I like kicking off large random wildcard runs with Stable Diffusion — this consists of the prompt for the image generation being comprised of random text pulled out of specific files. I’ve written about this several times so far:

Making Wild Images With Stable Diffusion

The X/Y Plot to Eliminate Some Stable Diffusion AI Art Models

Smile! Get Wild With Stable Diffusion — Wildcards and Facial Expressions

I’m going to cover how I set up my random prompt generator in ComfyUI, and then I’m going to discuss how I screwed something up and needed to fix it. Don’t try to implement what I did without reading all the way through.

Hello to SDXL and Goodbye to Automatic1111

Once SDXL was released I of course wanted to experiment with it. It seems just as disruptive as SD 1.5 was. But I can’t use Automatic1111 anymore with my 8GB graphics card just because of how resources and overhead currently are. I can, however, use the lighter weight ComfyUI.

I Want My Random Back!

Automatic comes with well-thought out features and exceptionally well-integrated extensions. I used the wildcard extension and the X/Y/Z plot to kick off runs for hundreds of images. The first thing I want back: something to replace my wildcard random prompt generation.

ComfyUI Community Has a Node For That

And yes, there are wildcard nodes out there. The one I’m going to discuss uses the Dynamic Prompt library. Now then, the Dynamic Prompt library is far beyond the simple wildcard extension I was using before. It’s very, very feature filled and super powerful. I was planning on migrating to it for Automatic1111, so it’s an easy step forward in ComfyUI.

Getting the Dynamic Node Working in ComfyUI

First, ensure you have a ComfyUI setup running. I’ll be working with SDXL in my flow. You can use this or a variety of videos on YouTube get yourself setup:

AI Art with ComfyUI and Stable Diffusion SDXL — Day Zero Basics For an Automatic1111 User

Next, once you’re looking at adding custom nodes to ComfyUI there is an essential first step: the custom node extension manager for ComfyUI. The best way to do this is to go into your ComfyUI directory and bring in the source code via the git command line:

cd custom_nodes/

git clone https://github.com/ltdrdata/ComfyUI-Manager.git

If ComfyUI is running, you’ll need to stop it, restart it, and refresh your ComfyUI web page.

When you start up ComfyUI, you’ll see that the control panel now has a new button: “Manager.” Click that and let’s find the dynamic prompts custom node package.

• Click Manager
• Search for (one word) dynamicprompts
• Look for result authored by adieyal.
• Install it.
• Just stop the ComfyUI server, don’t restart it yet.

Setup Your Wildcard Files

In your ComfyUI install directory, make a new directory called wildcards.

Put into that all the files you want to be consulted in your random prompt. For instance, I have a file called expression.txt that contains various facial expressions that I then refer to in the prompt as __expression__.

There are plentiful wildcard resources out there, even in CivitAI. And since you’re using the dynamic prompt library, you can download them with full directory hierarchy and organization. Or create your own.

Before you start up ComfyUI server again, go through and setup your initial set of wildcard files that you want to experiment with.

Wire Up Your New Random Dynamic Prompt Node

Now start the ComfyUI server again and refresh the web page.

You should have the ComfyUI flow already loaded that you want to modify to change from a static prompt to a dynamic prompt. Remember that you can drag and drop a ComfyUI generated image into the ComfyUI web page and the image’s workflow will be automagically loaded.

Look for your positive prompt. Right click near it to bring up the node menu. Select:

• Add Node ->
• Dynamic Prompts ->
• Random Prompts

A new string text box should be entered. You can now wire this up to replace any wiring that the current positive prompt was driving. For me, this was to both the base prompt and to the refiner prompt. Once wired up, you can enter your wildcard text. Start with something simple but that will be obvious that it’s working. For instance, if you have a wildcard file called fantasyArtist.txt with the names of various fantasy artists you like, something like:

Art by __fantasyArtist__

Will do.

Pros and Cons of Dynamic Prompts

Pros:

• Way more powerful than the standard Automatic1111 wildcard extension. It has a small language and my OCD can be comforted by having directories to organize everything in and use that structure in my wildcard syntax.

Cons:

• No live edits: it appears that a wildcard file is read once when referenced and not consulted again, unlike the A1111 wildcard extension, which readily adjusted to edits while Stable Diffusion was running. If you make essential edits, you’ll need to restart ComfyUI.
• No live file structure: just like it won’t pick-up edits to a file, it won’t pick-up changes to your file structure. If you add a new file, it will not recognize it until you restart ComfyUI.

Those Cons are a bit painful because restarting ComfyUI means reloading your models and such, which for me can take up to 10 minutes.

So measure twice and cut once here. And, well, it appears I should have measured three times or at least verified what I was doing was actually what I wanted…

…A Few Hundred Images Later

I’ve made a mistake in my initial setup here. I posted about this on Reddit, and I’m going to put bits and pieces of that post here.

So first on Reddit, u/rikkar posted an SDXL artist study with accompanying git resources (like an artists.txt file, just right for a wildcard run) — SDXL 1.0 Artistic Studies : StableDiffusion (reddit.com). I kicked off an overnight wildcard run pulling in artists from that text file in my random prompt, really excited by some of the images I see. I just can’t wait to discover the artist behind the images by analyzing the embedded metadata in the png (que depressing impending doom music).

I dump the metadata for a png I really like:

magick identify -verbose .\ComfyUI_01556_.png

Oh crap. There is no version of the generated prompt. Just the workflow including the wildcard prompt, but not what the random prompt generated. Anywhere.

I consider all my hundreds of now obscure wildcard generated images that I love and grudgingly mumble: “Makes sense…” because just the workflow to setup the run gets saved in the image, not the dynamic changes (like prompt generation) during the workflow run.

Skipping to the solution: first, an overview. Rather than embedding the generated prompt into the PNG, I’m going to generate a side-by-side file, a sidecar, that’s associated with the image. That sidecar file is going to contain the dynamic prompt. It’s going to look, name-wise, very similar to the PNG file, just that it will end with .txt instead of .png. Okay, ready to do it?

Saving Stable Diffusion Randomly Generated Prompts as Sidecar Files in ComfyUI

To solve this, I leaned on a very big node library for ComfyUI: WAS. Github location: WASasquatch/was-node-suite-comfyui: An extensive node suite for ComfyUI with over 180 new nodes (github.com) — I installed it via the custom node manager.

What I did:

• Via the ComfyUI custom node manager, searched for WAS and installed it.
• Restarted ComfyUI server and refreshed the web page. There was much Python installing with the server restart.
• First: (1) added IO -> Save Text File WAS node and hooked it up to the random prompt. Also: (2) changed my current save image node to Image -> Save Image WAS node.
• Bonus: image is now saved in a directory based on the day’s date. Nice for organization!
• Problem: no prompt text file saved -> I had to edit the path to begin with ./output instead of ./ComfyUI/output based on the relative location of where I run my server. YMMV. Now the text file is saved next to the image.

I like that idea of taking the prompt and making it a file prefix. To get this to work:

• I added a text truncation WAS node. I truncated from the beginning (more unique to me usually) for 96 characters.
• Ends up, WAS doesn’t fix-up filenames to be valid so you can’t go with a straight truncation. For instance, if you have something like (line art:1.5) the colon is going to prevent your file from being saved. So I had to add a Find & Replace text node. Luckily, this uses regular expressions so I backslashed the heck out of all the invalid characters I know: Find: [\\\/\:\*\?\”] Replace: -
• Now convert filename prefix in both the “Save Text File” and “Save Image” nodes to inputs and connect the output of Find and Replace to them.

Now I’ve got things nicely organized with obvious sidecar prompt files. Oy! Better than no prompts!

Original SDXL Flow Modified to Have a Random Prompt and Save to Sidecar

Well, I think that’s enough to make you dangerous with ComfyUI.