Transformer for neural machine translation (German → English)

Transformer encoder-decoder built from scratch, without Hugging Face, trained on Multi30k with BLEU evaluation against a seq2seq baseline.

Status: Research
Period: 2025
Area: Deep learning
Code: Code public
Stack: Python
PyTorch Lightning
Multi30k

German tokens
Positional encoding
Encoder (multi-head attention)
Decoder (masked + cross attention)
English tokens

Fig. 2 Data flow, left to right (top to bottom on small screens).

Problem

Build the Transformer from the paper, piece by piece, to understand every layer.

Approach

I implemented the Transformer encoder-decoder from scratch, including multi-head attention, positional encoding, and position-wise feed-forward layers, without Hugging Face.

Results

Trained on Multi30k (German → English) with BLEU evaluation, and compared against a seq2seq baseline.