Stable Diffusion

তিনটি কোর উপাদান

1. VAE Encoder/Decoder: 512×512 image ↔ 64×64×4 latent।
2. U-Net (Denoiser): latent-এ noise predict, text-এ conditioned।
3. Text Encoder: CLIP / T5 / OpenCLIP — prompt → embedding।

Inference Pipeline

text → CLIP encoder → embedding
random noise (64×64×4 latent)
   ↓ ×N steps: U-Net + scheduler + CFG
clean latent
   ↓ VAE Decoder
512×512 RGB image

Classifier-Free Guidance (CFG)

# একই step-এ conditional ও unconditional দুটিই predict
ε_cond   = unet(x_t, t, text_emb)
ε_uncond = unet(x_t, t, null_emb)

# guidance scale w (সাধারণত 7–9)
ε = ε_uncond + w * (ε_cond - ε_uncond)

HuggingFace Diffusers দিয়ে দ্রুত শুরু

from diffusers import StableDiffusionPipeline
import torch

pipe = StableDiffusionPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5", torch_dtype=torch.float16
).to("cuda")

img = pipe(
    prompt="a cinematic photo of a tiger in moonlight, ultra-detailed",
    negative_prompt="blurry, low quality, distorted",
    num_inference_steps=30,
    guidance_scale=7.5,
    height=512, width=512,
).images[0]
img.save("tiger.png")

Img2Img — ইমেজ থেকে ইমেজ

from diffusers import StableDiffusionImg2ImgPipeline
pipe = StableDiffusionImg2ImgPipeline.from_pretrained("...sd-v1-5").to("cuda")

out = pipe(prompt="ghibli style", image=init_img,
           strength=0.6, guidance_scale=7.5).images[0]

Inpainting

from diffusers import StableDiffusionInpaintPipeline
pipe = StableDiffusionInpaintPipeline.from_pretrained("...inpainting").to("cuda")
out = pipe(prompt="a red hat", image=img, mask_image=mask).images[0]

Conditioning Extensions

• ControlNet: canny, depth, pose থেকে structural control।
• IP-Adapter: reference image-এর style/identity inject।
• LoRA: ছোট adapter দিয়ে custom style/character।
• Textual Inversion: নতুন token শেখানো (~few image)।
• DreamBooth: নির্দিষ্ট subject পুরো fine-tune।

মডেল প্রজন্ম

Prompt Crafting টিপ

• Subject + style + lighting + camera + quality keyword।
• Negative prompt-এ বাজে artifact বাদ দিন।
• (word:1.3) দিয়ে weight বাড়ান।
• Seed fix করে reproducible variation।

অনুশীলন

সারসংক্ষেপ