Sixteen More Artists Studied in Stable Diffusion AI Art… and a Happy Accident
In my previous post (Stable Diffusion AI Art and Twenty-Five Artists | by Eric Richards | Nov, 2022 | Medium) I wrote about how I studied the effects of 25 different artists on one prompt. Well, I’m still studying, and learning a bit along the way. I came back to this after a pretty disastrous attempt at hypernetwork training on my face that led to CUDA tears. Stepping back into a comfort zone…
Context: I’m writing this at the start of the holiday season and I’ve been working on Christmas-time prompts. In this case, a prompt around a Christmas elf. That’s why you’ll see some odd ears in the pictures below — sometimes you get awesome elf ears and sometimes you get a big honking ear slapped onto the side of the head. Santa pops up occasionally, along with lots of Santa hats.
And sometimes exceptionally long necks. I don’t think that has anything to do with Christmas. Nor do the disfigured hands, which are for all the days of the year in Stable Diffusion land.
As someone who loves illustration, I am looking for generated compositions that seem to be telling a story or portraying a character in a dynamic way. Something more than a pretty lady with dead eyes staring at you.
As I look at artists, I’m looking for artists that Stable Diffusion has been trained on that impact the image in a significant way that has some aesthetic quality to it. So I’m poking the current V1.5 model with a sharp stick to see what it does.
First, Mike Mignola’s Happy Accident
Here’s a prompt I used that really impressed me. There’s something wrong with it. Well, at least one big thing. Can you spot it?
3D Vivid [handpainted:photograph:0.5] by Norman Rockwell and John Singer Sargent of pretty santa-workshop woman elven goddess, volumetric light, ornate leather dress, aesthetic, menacing, fantasy, chaotic, bokeh, intricately detailed, Symmetry, snowy, wet, Winter, Hyper-Realistic, Ultra Resolution, desolate, southern gothic, 8K, christmas, masterpiece [oil painting:hyperrealism:0.5] in the style of (Mike Mignola1.5)
I didn’t discover there was something wrong with that prompt until I realized I had been misspelling Michael Whelan’s name (one ‘e’ in the last name) and I was about to change artist when I noted, whoops, I had left out the colon between ‘Mike Mignola’ and the strength factor of 1.5, meaning that I wanted Mike Mignola’s style to be emphasized 150%. So fixed that by making it a proper (Mike Mignola:1.5) and re-ran a batch of images. Ah, now THAT is some Mike Mignola coming through.
Wow, okay. If I had done the proper 1.5 I would have moved on and never gone deeper into Mr. Mignola’s style. But instead I found an artist that created the kind of compositions I love. But what did that prompt originally do? Next batch: just say (Mike Mignola), which means a 1.2 emphasis.
Okay, a bit rich but not what we got from the original mistake. Assuming I know something about how that text is tokenized to be used by Stable Diffusion, I’m going to try 0.5 next — I think ‘Mike Mignola’ was extracted, the ‘1’ was dropped, and the .5 was taken to do 50% of Mr. Mignola. Let’s try (Mike Mignola:0.5):
Yeah, bingo. I think we found what the original run was now using proper syntax.
Lesson: if there is an artist’s style you really like but it’s doing too much or not enough, bring in the ability to focus how much of that artist you want in the prompt by using the form (artist name:number), where that number is below 1.0 to have less of the artist and above 1.0 to have more, basically as a percent where 1.0 is 100%, 0.5 is 50%, and 1.5 is 150%.
The Prompt Then the Artists
Here’s an example prompt of what I was using, pretty much the same from my last post:
3D Vivid [handpainted:photograph:0.5] by Norman Rockwell and John Singer Sargent of pretty santa-workshop woman elven goddess, volumetric light, ornate leather dress, aesthetic, menacing, fantasy, chaotic, intricately detailed, Symmetry, snowy, wet, Winter, Hyper-Realistic, Ultra Resolution, desolate, southern gothic, 8K, christmas, masterpiece [oil painting:hyperrealism:0.5] in the style of Guillem H. Pongiluppi
The last part, ‘Guillem H. Pongiluppi’ in this case, is what I’ll be changing to different artist names. It’s the primary artist with backup from two other artist’s styles, ‘Norman Rockwell’ and ‘John Singer Sargent’ in this case. You can decide who your backup artists are if you go through a similar exercise. My goal is semi-realistic portraits.
For the primary artist, I did experiment with strength of how much to apply the artist. Sometimes it is 150% to get more effect. Sometimes less.
I’m using the Automatic1111 Stable Diffusion web-ui environment. Oh, and a couple of notes. It took me a bit to realize that just because you find a fantastic artist doesn’t mean Stable Diffusion has been trained well (or at all) on that artist. Before starting a big batch of pictures, I’ve started switching to a 2nd tab of my local web-ui and typing in a simple prompt to see if Stable Diffusion produces anything that looks like it’s been trained on that artist.
When I call out an artist, it’s because what’s produced stands unique when blended with the other two artists. Maybe it’s the color and posing, maybe it’s the composition. It may look nothing like the artist’s original works because of the blending into something different.
Mike Mignola — Composition King
Happy accidents and all, Mike Mignola (of Hellboy fame) is my new favorite artist for interesting compositions. Even at 50% strength intriguing characters and compositions will pop-up in the image. The background typically has compelling elements as well, like dark thorny forests or spooky looking buildings.
As a reminder, I found Mike Mignola at 150% is full-on his style, and mixes well with other artists at 50% to 120%.
One thing that’s fairly unique for Mike Mignola-based output: all the faces, when there are multiple-people output, seem to look okay. Sometimes there are all scrunched and squiggled for other artists, but in general they look okay for Mignola.
Don Maitz
Don Maitz brings in heavy fantasy images with rich colors without being too saturated. The faces seem pretty reasonable. Ensemble pieces don’t hold together as well as Mignola.
Gaston Bussiere
Bussiere’s art was very on-target for this prompt, looking for a beautiful, powerful Elven goddess. His work has many mythical looking women in beautiful garb.
Roy Lichtenstein
Okay, this was a complete surprise, especially since I know a bit about Roy Lichtenstein’s style. Just for a grin, looking at a random collection of his images in a search result, I put him into the mix. It turned out awesome! The composition is interesting and the resulting artistic style is something I can’t quite name: it’s detailed but something about it seems flattened at the same time, reduced. Anyway, another happy surprise here.
Tomasz Alen Kopera
Kopera’s artwork is very organic looking — with structures and environmental elements melding with the portrait’s subject. Some of that comes out here with the environment around the images produced or the garb.
Victo Ngai
Victo Ngai’s art of fantastical scenes comes through with vivid detail here. The backgrounds don’t have much depth but they have detail and color schemes / transitions reminiscent of Ngai’s work.
NIXEU
Wow, I really like Nixeu’s art on ArtStation. The images going into Stable Diffusion result in a clear yet soft rendering of people with some very elegant clothing. The backgrounds are interesting but not particularly fantastic.
Virgil Finlay
I’ve been a fan of Virgil Finlay’s illustrations since I was in grade school. His art is fantastically detailed and I studied it for hours. It results in some nice compositions but not as cleanly handled by Stable Diffusion as say Mignola.
Krenz Cushart
I see Krenz Cushart coming up in the occasional prompt off of Lexica. The result here is soft dresses and presence in a quiet environment. Given the light-anime theme of Cushart’s art, it seems to match up.
Anders Zorn
The images produced with Anders Zorn have a soft painted feel to them, which makes sense given that Zorn’s artwork appears to be focused on Victorian-era living.
Anna Dittmann
Anna Dittmann’s influence is very strong in the composition, usually resulting in artistic images of flowers and swirls around a young lady’s face.
Krzysztof Porchowski Jr
Porchowski’s art itself is pretty focused on big huge muscular orcs and other heroic figures. In this case, you get some interesting portraits within plain environments. No majestic backdrops here.
Salvador Dali
For Salvador Dali, I suppose his strength could be increased to make things more surreal. What I see are serious looking women with a disaffected expression. I really like the one that has a line down the middle with actual differences on each side.
Jim Burns
Via Stable Diffusion, Jim Burns reference here mainly produced close-up portraits that look good, sometimes with interesting light and sometimes with interesting backgrounds. His art tends more towards science-fiction.
James Gurney
James Gurney’s style came through to create images of color-saturated ladies posing in various environments. There’s no kind of atmosphere attached to it and what Stable Diffusion produces is okay but not great. Gurney’s usual subject is fantastic beasts in fantastic locations (e.g., Dinotopia). Playing with the strength on his contribution might be interesting.
Pixiv
You see prompts referring to the aggregated site pixiv all the time. The images appear to be of anime situations. I’m surprised the result here didn’t come up with something more extreme / interesting.
I have more artists to cover that I’ve split out for a subsequent post, including one I think produces fantastic results along the line of Alphonse Mucha. Until next time.
