Training vs Inference
Training is studying for the exam. Inference is sitting the exam. This lesson makes that difference precise.
By the end of this lesson you will be able to
- explain the difference between training and inference in one sentence
- describe what happens inside a neural network in each phase
- recognise an inference request when you call a model on AWS
Inference is using a trained model to make predictions or decisions on new data it has never seen.
Training = learning
The model studies many examples and slowly adjusts itself to get better.
Inference = using what was learned
The finished model answers new questions. It is not learning anymore.
Training is studying for an exam 📚. Your knowledge changes as you study.
Inference is taking the exam 📝. You don't learn new material in the exam hall; you apply what you already know.
What happens inside the network?
Switch between the two modes. Notice which parts disappear during inference.
| Training | Inference |
|---|---|
| Learns from data | Uses learned knowledge |
| Changes the model's weights | Weights stay fixed |
| Expensive (hours to months on many GPUs) | Usually fast (milliseconds to seconds) |
| Needs a dataset | Needs only the new input |
| Done once, or periodically | Done every time a prediction is needed |
| Forward pass + backpropagation | Forward pass only |
The steps, written out
During training
Repeat millions of times.
During inference
No loss, no backpropagation, no weight changes.
Four examples
🍎 A child learning fruit
Training: you show thousands of pictures of apples, bananas and oranges.
Inference: shown a new picture, the child says “this is an apple”, using what they learned.
📧 Spam detection
Training: emails like “Win $1000” (spam) and “Meeting at 5 PM” (not spam).
Inference: “Claim your free iPhone” arrives → Spam (99.3%).
💬 ChatGPT
Training: massive amounts of text teach grammar, code, maths and reasoning.
Inference: you ask “What is inference?” and it generates an answer. No new learning happens.
🐶 Image classification
Training: thousands of cat, dog and bird images; the model learns features.
Inference: a new image → Dog (98%).
Inference is everywhere
| Application | Inference result |
|---|---|
| Face unlock | Recognises your face |
| Google Translate | Translates text |
| Netflix | Recommends movies |
| Amazon | Suggests products |
| ChatGPT | Generates responses |
| Tesla | Detects pedestrians |
| Medical AI | Predicts disease |
| Fraud detection | Flags suspicious transactions |
Inference on AWS
When you use Amazon Bedrock or Amazon SageMaker AI, inference means sending an input to a deployed model and receiving its output.
The API call is called an inference request, and the response is the inference result. You pay for inference each time you call the model; the provider already paid for the training.
“When I chat with an AI, it's learning from me in real time.” During a normal conversation the model's weights do not change: that's inference. A company may later use conversations as new training data, but that is a separate training step.
Key takeaways
- Training builds the model; inference puts it to work.
- Training = forward pass + loss + backpropagation + weight updates. Inference = forward pass only.
- Training is expensive and occasional; inference is cheap per request and happens constantly.
Check your understanding
1. Which step happens during training but not during inference?
Both phases run the forward pass. Only training measures the loss and sends corrections backward to change the weights.
2. You call Amazon Bedrock to summarise a document. This is…
You are using an already-trained model to produce output for your input.
3. Why is inference usually much cheaper than training?
Training repeats forward and backward passes over huge datasets many times. Inference is a single forward pass per request.