ট্রান্সফরমার

উচ্চ-স্তরের আর্কিটেকচার

Input → Embedding + Positional Encoding
   ↓
[ Encoder Block × N ]
   - Multi-Head Self-Attention
   - Add & LayerNorm
   - Feed-Forward (MLP)
   - Add & LayerNorm
   ↓
[ Decoder Block × N ]
   - Masked Multi-Head Self-Attention
   - Cross-Attention (encoder K,V)
   - Feed-Forward
   ↓
Linear + Softmax → Output token probs

Positional Encoding

Self-attention order-agnostic। তাই প্রতিটি position-এ আলাদা vector যোগ করা হয়।

PE(pos, 2i)   = sin(pos / 10000^(2i/d))
PE(pos, 2i+1) = cos(pos / 10000^(2i/d))

# Learnable position embedding-ও জনপ্রিয় (BERT, GPT)

Encoder Block — PyTorch

class EncoderBlock(nn.Module):
    def __init__(self, d=512, h=8, ff=2048, p=0.1):
        super().__init__()
        self.attn = nn.MultiheadAttention(d, h, dropout=p, batch_first=True)
        self.ln1  = nn.LayerNorm(d)
        self.ff   = nn.Sequential(nn.Linear(d, ff), nn.GELU(), nn.Linear(ff, d))
        self.ln2  = nn.LayerNorm(d)
        self.drop = nn.Dropout(p)
    def forward(self, x, mask=None):
        a, _ = self.attn(x, x, x, attn_mask=mask)
        x = self.ln1(x + self.drop(a))
        x = self.ln2(x + self.drop(self.ff(x)))
        return x

Decoder — Masked & Cross Attention

class DecoderBlock(nn.Module):
    def __init__(self, d, h, ff):
        super().__init__()
        self.self_attn  = nn.MultiheadAttention(d, h, batch_first=True)
        self.cross_attn = nn.MultiheadAttention(d, h, batch_first=True)
        self.ff = nn.Sequential(nn.Linear(d, ff), nn.GELU(), nn.Linear(ff, d))
        self.ln1, self.ln2, self.ln3 = nn.LayerNorm(d), nn.LayerNorm(d), nn.LayerNorm(d)
    def forward(self, y, enc, causal_mask):
        a,_ = self.self_attn(y, y, y, attn_mask=causal_mask)
        y = self.ln1(y + a)
        c,_ = self.cross_attn(y, enc, enc)
        y = self.ln2(y + c)
        y = self.ln3(y + self.ff(y))
        return y

তিন পরিবার

HuggingFace দিয়ে দ্রুত শুরু

from transformers import pipeline
clf = pipeline("sentiment-analysis")
print(clf("I love transformers!"))

# Translation
tr = pipeline("translation_en_to_de")
print(tr("How are you?"))

Training টিপ

• Warmup + cosine decay learning rate।
• Label smoothing 0.1।
• Mixed precision (fp16/bf16) — 2–3x দ্রুত।
• Gradient accumulation — বড় effective batch।

অনুশীলন

সারসংক্ষেপ