A Beginner’s Guide to Batch Optimization
The Beginner’s Guide to Batch Optimization
Have you been generating images in Stable Diffusion but don’t understand what batches are yet? That’s okay. In today’s article, I’ll explain to you what the batch size and batch count mean, how they differ, and what settings to use for them. Let’s get cooking!
What is Batch Size in Stable Diffusion?
In Stable Diffusion, batch size is the number of images that are processed at once by the software. It’s like the number of cookies that you place in the oven on the same pan at the same time.
What is Batch Count?
Batch count, on the other hand, is the number of image batches that Stable Diffusion will process/generate. It’s like having multiple pans of cookies but baking them in the oven one at a time.
So if you have a batch size of 5 and a batch count of 2, it’s the AI generation equivalent of having 5 cookies each on 2 pans, and baking one sheet of those tasty morsels at a time.
Batches Affect Your VRAM
But why do the number and size of batches even matter?
Because the batches that you generate will directly impact the amount of VRAM that your GPU must utilize to complete the image creation task. Larger batches will require more VRAM.
If the number of images per batch is set too high, you will run out of VRAM and Stable Diffusion will not generate the images.
That’s for when you are generating images. But batch sizes also make a considerable difference when you are training custom models.
Batches for Training Stable Diffusion Models
If you want to train your own custom model for Stable Diffusion, you’ll need to understand the relationship between batch size and VRAM usage.
When it comes to training the batch size determines how many images are processed and “learned” at one time.
So a bigger batch size will train on more images at once, and thereby make the training go faster. Think back to the cookie baking analogy. If you want to bake two dozen cookies, it will go a lot faster if you put all the cookies in the oven at the same time.
But more images per batch means more VRAM is needed to finish each batch. So the speed of your training will depend on how big of a batch that your graphics card can handle.
What is the Best Batch Size for Training a Stable Diffusion Model?
There is no one-size-fits-all when it comes to custom model training. The best answer is to test out the largest batch size that your GPU can handle without getting memory errors. If the results don’t look right, bring the step size down and re-test.
The long answer is that your optimum batch settings will depend on:
- How much VRAM you can utilize before maxing out,
- How many hours you have to train,
- The specific style or subject you are trying to train a model on
A GPU with 8 GB of VRAM can probably handle 5 or 6 images per batch. This should give you an average training time of 3 to 5 hours. In comparison, a batch size of 1 image could take around 12 hours to complete.
Should I Use Higher Batch Size or Batch Count?
But I’m assuming that most readers are not here to train a custom model. You just want to generate some cool AI art. In that case, should you use higher batch size or batch count? Does it affect the output images in any way?
Here’s the short of it:
- Increasing the batch size will generate images faster but use more VRAM
- Increasing the batch count will generate slower but won’t increase VRAM usage
There is some anecdotal evidence to suggest that small batches will create better looking images. However, I’ve yet to see confirmation of this from any developers working on AI software.
For the best AI image results, I recommend that you pick batches that don’t overload your GPU. If you only have 8 GB of VRAM, keep the batch size small.
I have a Nvidia GeForce RTX 3060 with 12 GB of VRAM and the max batch size I can run is 5 images. I almost always do small batch sizes and, if I find a prompt that looks good, I’ll run multiple batch counts.
Conclusion
Batch size and batch count determine how many images that Stable Diffusion will generate but in slightly different ways. The size of each will depend on how much VRAM you can utilize and how much time you have to wait for image generation.
Thanks for stopping by. If you found this article helpful, here are a few more you may like:
