The Quick Guide All About Negative Prompts and What They Do
You’ve probably seen talk about negative prompts in online Stable Diffusion communities or in videos. Some AI art enthusiasts claim it’s the secret ingredient to getting the perfect output image. But what dose negative prompt really do and how? In this guide, we’ll go over the ins and outs of negative prompts and how you should (and should not) add them to your Stable Diffusion prompts.
What is Negative Prompt in Stable Diffusion?
Negative prompt is a feature that lets you specify keywords you do not want in your image. The AI model will then try to push the prompt away from those words. Negative prompt is most commonly used to remove unwanted elements or styles from a prompt without having to modify the normal, positive prompt.
This feature can be extremely useful when something is appearing in your output images that cannot be removed through the use of positive prompt keywords.
For example, if you are trying to generate a landscape of of a city park but Stable Diffusion is also adding a lake or pond into the images. You can’t write “no lake” in the positive prompt, because the AI will simply see the word “lake” and add more water! Stable Diffusion does not understand human language syntax in that way.
So instead, you can add the word “lake” or “water” to the negative prompt. This will tell the AI model to avoid creating an image without bodies of water.
 |  |
| Prompt: city park, autumn, landscape, photo, 35mm lens (No negative prompt included) | Prompt: city park, autumn, landscape, photo, 35mm lens (Same seed, negative prompt of: water, lake) |
Why Don’t I See Negative Prompt in My Version of Stable Diffusion?
This is a user-created code modification that is available in a few distributions of Stable Diffusion, but not every repo of the AI software includes it. It is also not available in DreamStudio. You’ll likely need a local install of Stable Diffusion on your computer to access negative prompt.
The feature was first implemented by Automatic1111 in his Stable Diffusion Web UI repo and, due to it’s popularity, has spread to other interfaces and distributions. It’s also available in the NMKD Stable Diffusion GUI.
My Negative Prompt Words are Still Showing Up in Images. Why?
Just because you can tell Stable Diffusion to ignore a certain keyword, doesn’t mean it actually will! Negative prompts are not magic bullets that will instantly transform a regular prompts into a masterpiece…no matter how much internet gurus or discord jockeys claim it will.
Stable Diffusion operates by looking at your prompt keywords and inferring what an image with those keywords would look like based on the keyword-image pairs in it’s training dataset. If you add a subject word to the negative prompt that is strongly correlated to the positive prompt, Stable Diffusion may be unable to remove that subject.
An Example of Negative Prompt Not Working
For example, say you want to generate an image of a busy city street. But you add the words “cars”, “buildings”, and “peoples” to the negative prompt. Will Stable Diffusion just give you an image of a street alone? No.
Your images will likely still contain cars, people, and buildings (like the 4 generations above). Why? Because all of the data from which Stable Diffusion is inferring shows busy streets that have cars, people, and buildings in them. The strong correlation between images of “busy streets” containing “cars, people, and buildings” means Stable Diffusion has no choice but to include them in your output image because it doesn’t have a reference on how to paint a busy street without them.
In short: negative prompt is not a hard filter against a keyword. Rather, negative prompt uses conditioning to take the difference between your prompt and your negative prompt. What does that mean? Let me explain how the feature works so you know what to actually expect from using it.
What Does Negative Prompt Actually Do It?
So how is negative prompt actually working in the background? I’ll try to give you the layman’s version:
- The positive prompt guides the de-noising process to make your image look like the prompt. This is called conditioning.
- Unconditional conditioning is when the AI de-noises an image based on an empty prompt.
- The negative prompt replaces the unconditional conditioning (empty prompt) with the words in your negative prompt box.
- Stable Diffusion then looks at the difference between the 2 images (one with conditioning and one with unconditional conditioning) and changes the output to look less like the unconditional image.
So negative prompt is not telling Stable Diffusion “ignore pictures with these words altogether”; it’s really telling the AI “make the image look more like this (conditioning) and less like this (unconditional conditioning).
Don’t Use Wish Prompting
You’ll see hundreds of prompt tips and users online who tout their huge paragraphs of negative prompts, like:
(((deformed))), blurry, bad anatomy, disfigured, poorly drawn face, mutation, mutated, (extra_limb), (ugly), (poorly drawn hands), fused fingers, messy drawing, (mutated hands and fingers:1.5), (long body :1.3), (mutation, poorly drawn :1.2), black-white, bad anatomy, liquid body, disfigured, malformed, mutated, low res, bad anatomy, bad proportions, bad shadow, uncoordinated body, unnatural body, too many fingers, deformed fingers
And they’ll tell you that adding all of this filler will transform and beautify an image. But this is just magical thinking…it’s wishful prompting from people who don’t understand how a statistical mapping model works. Adding the words “deformed fingers” or “bad anatomy” isn’t going to make Stable Diffusion think “oh, I better make a good anatomical figure with the right amount of fingers this time”.
Because Stable Diffusion cannot think. It does not understand what “bad anatomy” means because the training data probably doesn’t have images tagged with that. I seriously doubt that someone went through all the billions of training images, looked for pictures of people with deformed fingers or hunched backs, and added tags to those pictures saying “bad anatomy”.
If the hands in your AI images look deformed, it’s not because Stable Diffusion looked at pictures of people with disfigured hands. It’s because hands are a very complex part of the human body to draw and the base model did not have enough training data to focus just on making realistic hands.
 |  |
| Prompt: a woman prating, clasped hands, photo portrait, 50mm lens, by Marta Bevacqua | With Negative Prompt of: deformed hands, poorly drawn hands, missing fingers, too many fingers, bad anatomy |
How to Really Use Negative Prompting
With that in mind, how should we use negative prompts?
Here are some best practices on how to implement negative prompts in a useful way, rather than throwing the kitchen sink at Stable Diffusion and hoping it develops human consciousness:
- Less is more
- Reactionary prompting
- Consider the training data
Less is More
First, don’t fill the negative prompt box with hundreds of words just because. It won’t make your images magically better, it will just waste your time and processing power.
Start small and only add a few words. In fact…
Reactionary Prompting
Don’t even use the negative prompt box the first time you try a specific prompt. Wait to see what is or is not wrong with the positive prompt first.
The best way to utilize negative prompts is by adding them only when there’s something in your image results that you want removed. Add negative keywords one by one and see how it affects the output images.
Consider the Training Data
Don’t think of Stable Diffusion like it’s a person who you are commissioning to make art. Think of it like a program that outputs images by inferencing what a string of keywords would look like together based on images it has seen before. Because that’s how it really works.
If you try to generate a picture of a woman’s hand but the fingers are messed up, then you need to understand why her hands are messed up. Stable Diffusion isn’t intentionally drawing bad hands. It just doesn’t have enough data about hands and fingers (at least, in the context you want) to make them realistic.
How can you remedy that? Perhaps use a custom model that has more training on human anatomy. Maybe even train a model on hands in context of portrait photos. The more data Stable Diffusion has about hands and how they should look, the more realistic hands should start to appear.
Conclusion
I hope this guide shows you how negative prompts can improve your AI art…and what it’s limitations are. If you found this article helpful, here are a few more you may like:
