How LLMs Work: Under the Hood
From raw text to a helpful assistant: the five steps that make a large language model work.
By the end of this lesson you will be able to
- explain how text becomes numbers a neural network can process (tokenisation and embeddings)
- describe the role of self-attention in the Transformer
- explain how an LLM generates text one token at a time
- outline the three training stages: pretraining, fine-tuning and RLHF
- explain why LLMs can give confident but wrong answers
Lesson 9 told the story with everyday examples. This one opens the machine and names the real parts.
1. What is a large language model?
A large language model is a neural network trained to predict the next word (technically, the next token) in a piece of text. That sounds too simple to explain everything ChatGPT, Claude or Gemini can do. But at massive scale, with the right architecture and training, “predict the next word, very well” turns out to be enough to produce reasoning, writing, coding and conversation.
Five steps take a model from raw text to a working assistant:
2. Step 1: Turning text into numbers
- Tokenisation: text is split into tokens (words or sub-word pieces), each mapped to an integer ID from a fixed vocabulary, often 50,000 to over 100,000 entries.
- Embedding: each ID is looked up in an embedding table and turned into a vector of a few hundred to a few thousand numbers. Tokens with related meanings end up with similar vectors. This is how the model represents meaning numerically.
Guess how “unhappiness” might be split into sub-word tokens. (One plausible split: un + happi + ness.) It shows why tokenisation isn't just “split on spaces”. Test your guesses in the toy tokeniser.
3. Step 2: The Transformer architecture
Once text is a sequence of embedding vectors, it flows through the Transformer. First a positional encoding is added, since token order would otherwise be lost. Then the sequence passes through N identical blocks stacked on top of each other; frontier models use anywhere from a few dozen to over a hundred.
Each block has two main parts:
- Multi-head self-attention lets every token gather context from every other token (step 3 below).
- A feed-forward network, a small neural network applied to each position, adds modelling capacity after attention has mixed in the context.
“Add & layer norm” steps surround each part. They add a shortcut connection (which helps the learning signal flow through very deep stacks) and rescale the values for stable training. After the final block, a linear layer projects back to the size of the vocabulary, and softmax turns that into a probability for every possible next token.
4. Step 3: Self-attention, the key mechanism
Every token looks at every other token and decides how much to “attend” to it, then blends in information accordingly. This is how the model resolves ambiguity. In “The animal didn't cross the street because it was tired”, a human knows “it” means the animal. Self-attention learns the same thing: a strong weight from it back to animal.
Mechanically, each token produces three vectors: a Query, a Key and a Value. A token's Query is compared with every token's Key to produce relevance scores; softmax turns those into weights; the weights blend the Value vectors. Multi-head attention runs several of these in parallel, each free to learn a different relationship (one head might track grammar, another the topic).
5. Step 4: Generating text, one token at a time
An LLM doesn't write a whole sentence in one shot. It predicts a probability distribution over the entire vocabulary for the single next token, picks one (usually by sampling, occasionally just the most likely), appends it, and feeds the longer sequence back in. This loop is called autoregressive generation.
Because the model usually samples from the probabilities rather than always picking the top one. “mat” may win most of the time, but sometimes “floor” is chosen. A setting called temperature controls how adventurous this sampling is.
6. Step 5: How an LLM is trained
Pretraining
The model reads enormous amounts of text (much of the public internet, books, code) and learns to predict the next word, over and over. It absorbs grammar, facts and reasoning patterns. The result, a base model, is not yet a helpful assistant: ask it a question and it may just continue your text.
Supervised fine-tuning (SFT)
The base model is trained further on a smaller set of curated examples, instructions paired with good responses, so it behaves like an assistant instead of just continuing text.
RLHF
Reinforcement Learning from Human Feedback. Human raters compare and rank several responses to the same prompt. Those preferences train the model to favour helpful, honest and safe answers. (This is the reinforcement learning from lesson 2.)
7. Why LLMs sometimes get things wrong
An LLM predicts plausible next words; it doesn't look up verified facts. So it can produce text that sounds fluent and confident but is wrong. This is called hallucination.
- The model has no built-in fact-checker; it generates whatever continuation its training made statistically likely.
- It has no memory of “truth” beyond patterns in its training data, and that data has a cutoff date.
- It can be wrong in exactly the same confident tone as when it is right, which is why human review matters for high-stakes use.
That's why real-world systems often give the model search tools, or ground its answers in retrieved documents using RAG (retrieval-augmented generation). The model can then check its work against a real source instead of relying only on memorised patterns.
8. Key terms
- Token
- A chunk of text (often a sub-word) the model treats as one unit.
- Embedding
- A vector of numbers representing a token's meaning in a form the model can compute with.
- Self-attention
- The mechanism that lets each token weigh how relevant every other token is.
- Parameter
- A learned number inside the model (a weight or bias). Modern LLMs have billions.
- Context window
- The maximum number of tokens the model can consider at once.
- Pretraining
- The first, large-scale phase: predicting the next word across huge amounts of text.
- Fine-tuning
- Extra training on a smaller, curated dataset to specialise or align behaviour.
- RLHF
- Reinforcement Learning from Human Feedback: tuning with human preference rankings.
- Hallucination
- Fluent text that is factually incorrect or made up.
- Inference
- Running a trained model to generate output, as opposed to training it.
9. Discussion questions and activities
- Pick a sentence with a pronoun (“it”, “they”) and guess which earlier word the model attends to most. Then test it on a real chatbot.
- Break 3 or 4 unusual or made-up words into plausible sub-word tokens by hand. Why isn't tokenisation the same as splitting on spaces?
- Ask a chatbot about events after its likely training cutoff. How does it respond? Connect this to hallucination.
- Debate: should an AI assistant always say “I don't know” when unsure, or is some hallucination an acceptable trade-off for fluency? Would your answer change for a medical app versus a creative-writing app?
- Give the same prompt to two different models or model sizes. What do the differences suggest about scale and training data?
Summary
- An LLM turns text into numbers (tokens and embeddings) and processes them through stacked Transformer blocks built around self-attention.
- It generates a response by repeatedly predicting one next token.
- It becomes a helpful assistant through pretraining → fine-tuning → RLHF.
- Its core skill is plausible continuation, not verified lookup, so it can sound confident while being wrong.
Check your understanding
1. What does the softmax at the end of the Transformer produce?
The final linear layer scores every vocabulary token; softmax turns the scores into probabilities.
2. Which stage turns a base model into something that follows instructions?
SFT trains on instruction/response pairs. RLHF then refines it using human preferences.
3. How does RAG reduce hallucinations?
Retrieval gives the model real sources to base its answer on, instead of memory alone.