এমবেডিং

One-hot বনাম Embedding

Word2Vec (Mikolov et al., 2013)

দুটি স্থাপত্য: CBOW (পার্শ্ববর্তী শব্দ থেকে কেন্দ্র) এবং Skip-gram (কেন্দ্র থেকে পার্শ্ব)। জাদু: king − man + woman ≈ queen।

from gensim.models import Word2Vec
sents = [["আমি","পাইথন","শিখছি"], ["সে","বাংলা","ভালোবাসে"]]
m = Word2Vec(sents, vector_size=100, window=5, min_count=1, sg=1)
print(m.wv.most_similar("পাইথন"))

GloVe, FastText

• GloVe: global co-occurrence matrix factorization।
• FastText: subword (char n-gram) → OOV/বাংলা morphology-তে ভালো।

Keras Embedding Layer

from tensorflow.keras import layers
emb = layers.Embedding(input_dim=vocab_size, output_dim=128, mask_zero=True)
x = emb(token_ids)   # (B, T) → (B, T, 128)

Pretrained embeddings load

import numpy as np
matrix = np.zeros((vocab_size, 300))
for w, i in word_index.items():
    if w in glove_dict:
        matrix[i] = glove_dict[w]

emb = layers.Embedding(vocab_size, 300, weights=[matrix], trainable=False)

Contextual Embeddings — ELMo, BERT, …

Word2Vec/GloVe-এ "bank" সবসময় একই vector। BERT-এ "river bank" আর "money bank"-এর vector আলাদা — context অনুযায়ী।

from sentence_transformers import SentenceTransformer
m = SentenceTransformer("paraphrase-multilingual-MiniLM-L12-v2")
v = m.encode(["আমি বাংলা ভালোবাসি", "I love Bengali"])
# cosine similarity → ~0.85

Visualization — t-SNE / UMAP

from sklearn.manifold import TSNE
import matplotlib.pyplot as plt
red = TSNE(2).fit_transform(vectors)
plt.scatter(red[:,0], red[:,1])
for i, w in enumerate(words): plt.annotate(w, red[i])

অনুশীলন

সারসংক্ষেপ