Steer with a picture

Try to art-direct with adjectives: "a cat on the LEFT, a lighthouse in the BACK, sun in the top-right corner." Generate it a dozen times and the cat wanders, the lighthouse floats, the sun does whatever it likes. The model treats layout words as vibes, not coordinates.

Chapter 1 told you why. The text encoder turns your prompt into a bag of numbers — it captures what you asked for beautifully, but geometry mostly doesn't survive the trip. "Left", "behind", "top-right" arrive as faint suggestions, not a floor plan. Crank guidance and you don't get better placement; you get a more saturated version of the same misplacement.

So when you care about where, stop adding adjectives and add a second input channel: an image. Three tools, in order of how much of the source they keep:

• img2img — hand over a whole picture; the model repaints all of it, more or less faithfully.
• Inpainting — hand over a picture and a mask; only the masked region gets repainted.
• ControlNet — hand over just the structure of a picture; the model keeps the bones and repaints everything else freely.

Remember Chapter 1's loop — guess the noise, subtract a bit, repeat. Plain txt2img starts that loop from step 0: pure static, nothing but a seed. img2img starts it somewhere else. It takes your image, adds noise partway back — erasing some of it — and then runs the exact same denoise loop from that half-erased state.

One dial governs the whole thing: strength, i.e. how much of your image gets erased before the repaint begins.

• 0.2 – 0.35 — barely erased. Composition and palette survive; you're just repainting the surface in a new style. This is Chapter 4's anchor move.
• 0.5 – 0.65 — the gist survives: rough layout and subject, everything else prompt-side.
• 0.75+ — only a rumor of the original is left; at 1.0 you're back to txt2img.

Here's the quiet gotcha: img2img actually runs about strength × steps of the loop, not all of them — because it skipped the early ones by starting partway. So very low strength barely does anything by design. Drag the dial and watch the repaint zone shift:

Three recipes that make img2img worth a permanent slot in your workflow:

• Sketch → render. Block out the composition yourself — MS-Paint level is fine, coloured blobs where things go — then run it at strength ~0.6. You supply the where; the model supplies the finish. This is the single best answer to "the model won't put things where I want."
• Style transfer. A photo you like + your Chapter 4 style block + a style LoRA, at strength ~0.4. The photo keeps its bones; your look paints over it.
• Iterate on a winner. Take your best output, feed it straight back at strength ~0.3 with a slightly tweaked prompt. The safest "same but better" move there is — you nudge one thing without rerolling the whole composition.

img2img repaints the whole frame. Often you only want to fix one corner — the classic broken hand, an ugly sky, a photobomber in the background. That's inpainting: you paint a mask over a region, and the loop runs as normal, except at every step only the masked pixels are allowed to change. Everything outside the mask is clamped back to the original, step after step. The rest of the picture comes out bit-identical.

Paint a mask below — drag across cells, or tap a preset — then hit Generate. Only the masked region is repainted; watch the unmasked pixels stay exactly put:

Two craft notes that separate clean inpaints from obvious ones:

• Mask a little generously. Overshoot the edge of the thing you're replacing. The model needs elbow room around the boundary to blend the new region into the old one — a tight mask leaves a visible seam.
• Describe the whole scene, not just the patch. The prompt for an inpaint should read like a caption for the finished picture (or at least the region in context), so the repaint matches its surroundings. Prompt only "a hand" and you'll get a hand that ignores the arm it's attached to.

Outpainting is the same trick pointed outward: extend the canvas past its edges, mask the new blank margin, and let the model continue the picture into it. Same clamp, same loop — you're just inpainting the outside. DiffusionBee ships img2img and inpainting built in; the buttons move between versions, so treat the exact UI loosely.

Here's the job img2img can't do. You want the exact pose and layout of a source image, but a completely different picture — new palette, new medium, new subject skin. Low strength keeps the structure but also keeps the old look; high strength changes the look but erases the structure right along with it. There's no strength value that keeps one and drops the other.

ControlNet splits them apart. Plain version first: it extracts just the bones of your source — the edges, or a stick-figure pose, or a near/far depth map — and pins the denoise loop to those bones, while the prompt and your LoRAs repaint everything else freely. It's tracing paper: your lines stay put, the drawing on top of them changes completely.

Pick a control type, a style, and a weight, and watch the same bones become three different drawings:

Now the one paragraph of machinery, since you've earned it. A ControlNet is a trained copy of the U-Net's encoder half that reads the control map and, on every denoise step, nudges the guess toward "…and match these bones." It's Chapter 1's loop with a chaperone whispering the structure at each step. Crucially, the model's original weights stay frozen — the ControlNet only adds. That's why ControlNets are separate downloads that compose with any prompt and any LoRA stack (Chapter 3): structure and style are independent add-ons you mix at will.

Different maps keep different amounts of the source. You pick the one that encodes only what you care about — everything it doesn't encode, the prompt gets to invent.

The rule of thumb: pick the map that keeps only what you care about. The less a map encodes, the more room the prompt has to work. OpenPose throws away everything but the skeleton — perfect when only the pose is sacred. Canny keeps every edge — perfect when the linework is the point, and a straitjacket when it isn't.

Two dials shape how hard the trace holds — and you met the logic behind both in Chapter 2.

• Weight — how firmly the trace pins the loop. Around 1.0 the output obeys the map; below 0.7 it's a gentle suggestion the prompt can overrule; push it too high and you get a stiff, embossed look with the map's own lines ghosting through the final texture. (You saw exactly this on the weight slider above.)
• Start / end steps — when the chaperone is in the room. Chapter 2's insight: early steps decide composition, late steps decide texture. So end the ControlNet's influence at ~60–80% of the steps — it locks the layout in the early passes, then lets go so the final passes polish freely. That's the practical cure for stiffness.

And here's the payoff: ControlNet stacks with everything else in this book. Structure from the ControlNet + style from a LoRA (Ch.3) + consistency from Chapter 4's frozen recipe = the whole guide firing in a single generation. Structure, look, and reproducibility, each on its own dial.

All three tools are the same generate function under the hood. They only change two things: where the loop starts, and who's chaperoning it. txt2img starts from static with no chaperone; img2img starts from your half-erased image; ControlNet starts from static again but hands the loop a control map. Flip between the languages — the shape is identical.

You now have four ways to steer, and they don't compete — they answer different questions. The decision comes down to how much of a source you want to keep:

That's the last wheel. You started with pure static and a single seed, learned the knobs that shape it, the patches that bend it, a style that's genuinely yours — and now the pictures that place it. Words, knobs, patches, and pictures: you have every steering wheel the model offers. Go point it exactly where you want.