๐Ÿ™Œ Hands on: a Stable Diffusion prompting project

๐Ÿง‘โ€๐Ÿ’ป Capturing the essence of Poolsuite into a prompt

Let's get our hands dirty! We'll take on a small project to go over the whole workflow of creating a prompt, iterating it and generating good images with Stable Diffusion.

๐Ÿ– Backstory: what is Poolsuite?

Here at PromptHero we're huge fans of Poolsuite. Poolsuite is an internet project that brings retro aesthetics back to the present: everything they do, from their websites' UI, to the images they use, to the texts they write is carefully crafted to give users a unique experience and make them feel like they're living a few decades back in the past.

Here are some pictures from their Instagram page, so you get an idea of what Poolsuite represents:

We love their vibe, so let's try to replicate their aesthetics using Stable Diffusion and prompt engineering!

What makes up the image?

The first thing we want to do is to come up with a set of words that best describe what we're trying to generate. How would you describe Poolsuite to a friend?

I would say the images above represent the glory of the 70s through 90s, the life of high-end luxury that the elite lived in places like Ibiza and Miami. Some words and concepts that come to mind would be: vintage, retro, mansions, palm trees, beach, martini cocktails, champagne, disco music, tennis, cigars, bikinis, emerging technology (brick phones, compact discs...) A golden era. Old money. Executives living their best life.

๐Ÿ‘€ Pro tip: Where do I get good words to describe what I'm going for?

Here are a couple of tricks that might help getting you ideas for words to describe the image you want to create.

The first one, obviously, is to browse PromptHero for images similar to the one you want to create, and draw ideas from the prompts used to create them. In our case, a good starting point could be searching for the word "pool".

The second trick is to use the "CLIP interrogator" feature in Automatic1111, which gives you its best guess at what prompt could generate that particular image. It's far from perfect, so we're only using it as inspiration. (If you don't use Automatic1111, here's an alternative to use the CLIP interrogator feature)

In the img2img tab on Automatic1111, drop an image into the img2img area and hit the "Interrogate CLIP" button:

Don't click "Generate" or anything else. Just "Interrogate CLIP". After a few seconds, it will write in the prompt box area the prompt it thinks it best describes what the image is:

This gives us a few more ideas: bathing suits, large pool, "by Slim Aarons" โ€“ very good ideas that we might want to try later on!

And with that โ€“ congratulations! We got ourselves a starting point. Keeping these words nearby will come in handy when writing our prompt.

How do we start writing the prompt? What's the subject?

We need to have a starting point for our prompt. If I had to say what most of the images above have in common, I'd say that would be a pool. That's going to be our subject. So let's just start with:

"pool in the garden of a mansionโ€

That's... disappointing. It's accurate, in the sense that these are pictures of a pool in a garden, but we're pretty off from the style we want to replicate. Remember, everything left unsaid will not be rendered. So let's keep adding details, we'll try adding the word vintage next:

โ€œpool in the garden of a vintage mansionโ€

Not a bad start, huh? It's definitely not perfect, but we're getting close.

๐ŸŽฏ Be precise

Let's keep walking down this path and sprinkle some Poolsuite magic over our prompt. What else does Poolsuite represent? The more precise you are in the prompt, the better the outcome. Don't expect the AI to fill in the gaps for you.

We get to the third iteration of our prompt by adding a few words from our vocabulary above:

โ€œpool in the terrace of a vintage mansion, vintage furniture with martini cocktails, gold, plants; luxury vibes, old money dynasty color paletteโ€

I think we're getting closer! There is still more work to do, though.

๐ŸŽจ Adding styles

Something all Poolsuite's pictures have in common is the washed-out effect, as if they had been taken with an old film camera. Words that come to mind are analog photography, film, lomo, 35mm...

As always, you can head to PromptHero to look for inspiration for prompts!

If we tweak our prompt a little and add the "35mm lomo photography" style, we get to the final iteration of our prompt:

"35mm lomo photography of a pool in the terrace of a vintage mansion, vintage furniture with martini cocktails, gold, plants; luxury vibes, old money dynasty color palette"

Looking good!

Remember what we learned in Prompt Engineering 101? The prompt we just wrote pretty much follows that structure โ€“ but you can play with it and keep changing things until you get a result you like.

[art medium], [main objects], [attribute], [expression], [key light], [detailing], [camera shot], [media], [engine], [artist]

๐ŸŒŠ What to try next โ€“ and the limits of Stable Diffusion

I would say this is pretty much as close as we can get to the pool in the garden ร  la Poolsuite vibe using Stable Diffusion alone. You're welcome to keep trying ideas, though!

  • One promising idea we got earlier could be adding the modifier "by Slim Aarons" at the end. I think Stable Diffusion would yield great aesthetics with it!
  • Another idea is to do what we already learned in Midjourney: keep generating images with a prompt you like until you hit a seed that gives you an outstanding result. Then, fix the seed and keep tweaking the prompt.

Of course, we're missing a bunch of things in the image, especially people. As we've already learned, rendering anatomically correct people is one of Stable Diffusion's limitations, and composing things (generating both people + Poolsuite style images) is another limitation, so we're only able to generate the overall vibe for now.

In just a few lectures, we'll cover model fine-tuning and Dreambooth, techniques which will allow us to create even better Poolsuite-like images, but for now this is the best Stable Diffusion can get us!

๐Ÿง‘โ€๐ŸŽจ Tweaking styles

As we mentioned a few lectures ago, prompts are infinite image generators. What this means is you can "parameterize" parts of the prompt to replace certain words and generate completely different images.

Let's try tweaking the style: we'll replace 35mm lomo photography with victorian illustration to get the following prompt and result:

"victorian illustration of a pool in the terrace of a vintage mansion, vintage furniture with martini cocktails, gold, plants; luxury vibes, old money dynasty color palette"

Radical change, huh?

While this doesn't really represent the overall Poolsuite vibe, it shows the power of prompts as infinite image generators. You can take the prompt and change the style for oil painting, digital painting, 3D render... anything you can think of!

You could also tweak other parts, and make the scene take place in a garden instead of a terrace, for example.

We ourselves have played with this particular prompt quite a bit! We even made a Poolsuite generator in PromptHero, based on everything I've mentioned:

Don't be afraid to tweak the prompt and keep playing with it! Get inspiration from other PromptHero prompts, or try your own ideas!

You'll keep progressing on your prompt engineering journey in the following modules, learning new tricks as you go down this wonderful path โ€“ but what you know now is already extremely powerful. Please take a few moments to explore the tools you already know, and unleash your creativity by tweaking this prompt!

โœ… Before you move on

If you've followed the steps above, I'm sure some of the images you've created look absolutely great! Please share them with us on Twitter โ€“ and let the whole world see your creations!

Complete and Continue