In the spirit of full disclosure, the content of this post is heavily cribbed from this post on Reddit. However, as we've seen, the Internet is not forever. It is entirely possible that a wealth of knowledge could be lost at any time due to any number of reasons. Because I have found this particular post so helpful and find myself coming back to it over and over, I thought it would be appropriate to share this method of creating an inpainting model from any custom stable diffusion model.
Inpainting models, like the name suggests, are specialized models that excel at "filling in" or replacing sections of an image. They're especially good at decoding what a section of an image should look like based on the section of image that already exists. This is useful when you're generating images and only a small section needs to be corrected, or if you're trying to add something specific to an image that exists.
So how is this done? With a model merge. Automatic1111 has an excellent model merging tool that we'll use. Let's assume that you have a custom model called
my-awesome-model.ckpt that is based on
In the A1111 Checkpoint Merger interface, follow these steps:
- Set "Primary model (A)" to
- Set "Secondary model (B)" to
- Set "Tertiary model (C)" to
- Set "Multiplier (M)" to
- Set "Interpolation Method" to
- Give your model a name in the "Custom Name" field, such as
And bazinga! You have your own custom inpainting stable diffusion model. Thanks again to /u/MindInTheDigits for sharing the process.