How a language model gets made
Three phases turn raw text into something you can chat with. This lab lets you drive every one of them by hand, on your own input, in the browser. Click a phase to jump straight there.
Whysimulate instead of just explaining
The words "attention" and "reinforcement learning" stay abstract until you've watched a number change because you changed something. Every panel in this lab does a real computation — real softmax, real dot products, real cross-entropy — just at a toy scale (8 dimensions instead of 12,288) so you can see every value at once instead of drowning in a 175-billion-parameter matrix.
What you'll be able to answer afterward
- What actually happens, numerically, when a sentence goes into a transformer
- Why attention is "queries, keys and values" and not just one vector
- Why we need positional encoding at all when attention itself is order-blind
- What a residual connection and layer norm are protecting against
- Why pretraining is "just" next-token prediction, and why that's enough to learn a lot
- What SFT, RLHF (reward model + PPO), DPO and GRPO each optimize, and why teams pick one over another
Tokenizer
Models don't read letters or words — they read a fixed vocabulary of tokens, learned by finding the most common repeated chunks of text. This is a real byte-pair-encoding (BPE) loop, trained live on whatever you type.
Token embeddings
Each token in the vocabulary owns one learned vector — its "meaning" as a list of numbers. Real models use ~4,000–12,000 numbers per token; here we use 8 so you can see every one. These particular numbers are random (untrained) — pretraining is exactly the process that tunes them into something meaningful.
Positional encoding
Self-attention (next section) looks at every token at once with no sense of order — "cat sat on mat" and "mat on sat cat" would look identical to it. So before attention runs, we add a wave-pattern vector for each position that lets the model tell position 1 from position 7.
Each dimension i oscillates at a different frequency — low dimensions wiggle fast (distinguish neighboring positions), high dimensions wiggle slow (distinguish far-apart positions). Added together across all 8 dimensions, every position ends up with a unique fingerprint.
Self-attention
This is the mechanism that lets a token gather context from every other token. Each position emits a query ("what am I looking for?"), and every position (including itself) offers a key ("what do I contain?") and a value ("what do I actually give you if you attend to me?").
Dividing by √d_k keeps the dot products from growing huge as dimensions increase — without it, softmax saturates and gradients vanish. Softmax turns each row into a probability distribution: "how much should this token attend to every other token."
Each head learns its own W_Q/W_K/W_V, so it can specialize — one head might track "which noun does this adjective describe," another "what's the previous verb." Their outputs get concatenated and mixed back down with one more matrix, W_O.
Residual connections, layer norm & feed-forward
Attention's output doesn't replace the token's representation — it's added to it. Then two more ingredients finish the "transformer block": normalization to keep numbers stable, and a small per-token neural net to add nonlinear processing.
Without this "+X", stacking many layers would make gradients vanish or representations drift arbitrarily far from the input. The residual path is a direct, always-available shortcut for information (and gradient) to flow through every one of the network's layers unchanged if that's the best option.
Re-centers and re-scales each token's vector to mean 0, variance 1 (then lets the model learn a new scale γ and shift β). Keeps values from exploding or shrinking as they pass through dozens of layers.
Attention is the only place information moves between positions. The feed-forward network is where most of a transformer's parameters live, and it does per-token computation — think of attention as "gather the right context" and the FFN as "now think about it."
Stacking blocks into a model
One block gathers context and thinks about it once. A real model chains dozens of them, each refining the representation further — early layers tend to pick up on syntax and local patterns, later layers on more abstract, task-relevant meaning.
Depth vs. widthtradeoff
| Going deeper (more layers) | Going wider (bigger d_model) |
|---|---|
| More sequential refinement steps; better at multi-step composition | More capacity per step; more room to represent nuance at once |
| Harder to train — needs residuals/normalization to avoid vanishing signal | Quadratic parameter cost in d_model; attention cost grows too |
| Slower at inference (strictly sequential through layers) | More parallelizable per layer on modern hardware |
Output layer & sampling
After the last block, the final position's vector is projected onto every token in the vocabulary, producing one score (logit) per token. Softmax turns those into probabilities. How you turn probabilities into an actual next token is a design choice — try the controls below.
| Temperature | Divides logits before softmax. Low (→0) sharpens the distribution toward the top choice (deterministic, repetitive); high (>1) flattens it (more random, more diverse, more mistakes). |
|---|---|
| Top-k | Zero out every token outside the k highest-probability options, then renormalize. Hard cutoff on how many candidates are even eligible. |
| Top-p (nucleus) | Keep the smallest set of top tokens whose probabilities sum to p, then renormalize. Adapts the candidate pool size to how confident the model is. |
Pretraining
Every mechanism above starts with random weights. Pretraining is the process that tunes billions of numbers so the forward pass you just explored actually produces sensible text.
The objectivenext-token prediction
Take huge amounts of raw text. For every position in every document, hide what comes next and ask the model to predict it from everything before. Compare the model's predicted distribution to the actual next token using cross-entropy loss, and nudge every weight slightly in the direction that would have made the correct token more likely.
That's it — no labels, no human annotation, just "predict what comes next." It works because doing this well at scale forces the model to implicitly learn grammar, facts, reasoning patterns, and style, since all of those help predict text.
Scaling lawsthe empirical headline
Loss falls predictably as a power law in three resources: model parameters (N), training tokens (D), and compute (C). Given a fixed compute budget, there is a roughly optimal split between "make the model bigger" and "train on more data" (this is the intuition behind results like Chinchilla) — over-invest in either alone and you leave loss on the table.
From text-predictor to assistant
A pretrained model is very good at continuing text, but it isn't automatically a good assistant — asked a question, it might continue with more questions (that's what usually follows a question in web text). Post-training reshapes its behavior.
Side-by-side
| Method | Needs a separate reward model? | Needs on-policy rollouts? | Relative compute / complexity | Typical use |
|---|---|---|---|---|
| SFT | No | No | Low — same recipe as pretraining, on curated data | Teach the response format & baseline behavior |
| RLHF (PPO) | Yes — trained on human preference pairs | Yes — generate, score, update, repeat | High — 3-4 models in memory (policy, reference, reward, critic) | General alignment; strong but expensive, unstable to tune |
| DPO | No — preference pairs used directly in the loss | No — trains offline on a static preference dataset | Low-Medium — one model, one loss function | Cheaper, more stable alternative to RLHF for preference alignment |
| GRPO | No critic network — reward can come from a rule/verifier or model | Yes — samples a group of completions per prompt | Medium — no critic to train, but needs group sampling | Reasoning/math/code tasks with checkable rewards |
Quiz
Fifteen questions spanning everything above. Click an answer to check it.
Full transformer architecture
This is the whole route in one view: your text is tokenized, embedded, given position, passed through stacked blocks, projected to logits, and sampled back into new text. Change the prompt or generation controls here and the rest of the lab updates with it.