Variational Autoencoders (VAEs)
Squeeze data down to its essential features, organise them on a smooth map, then pick any point to create something new.
By the end of this lesson you will be able to
- explain what an autoencoder does, and how a VAE improves on it
- describe latent space in plain words
- explain why a VAE can generate new data, and how it compares with a GAN
First, understand the autoencoder
Imagine a large image. An autoencoder tries to:
- Compress it into a small hidden representation, and then
- Rebuild the original image from that compressed information.
A bit like ZIP compression, except the network learns for itself what is worth keeping.
Instead of storing every pixel of a photo, you describe the person using only height, hair colour, face shape and eye type. That compact summary is called the latent space representation.
A VAE has two main parts
1. Encoder: the compressor
Turns the input into compressed hidden features. A cat photo becomes something like: fluffy, small ears, white fur, round eyes. This is the latent vector.
2. Decoder: the rebuilder
Takes the latent representation and recreates data from it: a cat image, a face, audio, and so on.
So what makes a VAE special?
A normal autoencoder only learns to compress and rebuild. Its hidden space can be messy, so picking a random point often decodes into garbage. A VAE does something smarter: it organises the latent space smoothly.
Instead of storing faces randomly, a VAE arranges them on an organised map: young faces on the left, older on the right; one feature changing top to bottom. Similar things sit close together, dog images near dog images and cat images near cat images. Now the AI can create new faces by picking any point on the map.
Why probability distributions?
This is the part beginners find confusing, so let's simplify. A VAE does not store one exact compressed value. It stores a range, usually as a Gaussian (bell curve).
Autoencoder says
“Face age = exactly 25.”
VAE says
“Face age is likely around 25, give or take.”
That flexibility fills in the gaps on the map, so you can pick random points, feed them to the decoder, and generate completely new, realistic samples: a new face, a new anime drawing, new handwriting. The “Sample” button above does exactly this, drawing from a bell curve centred on the map.
GAN vs VAE
| GAN | VAE |
|---|---|
| Learns by competition | Learns by compression + reconstruction |
| Generator vs discriminator | Encoder + decoder |
| Usually sharper images | Usually blurrier images |
| Hard to train | More stable |
| Excellent realism | Better-organised latent space |
GAN is like…
a fake artist trying to fool a police inspector.
VAE is like…
compressing many face patterns into a smart, organised feature map, then creating new faces from that map.
The encoder extracts hidden “DNA-like” features. The latent space stores those instructions. The decoder uses them to build the organism, or in our case the image.
Why VAE images can look blurry
A VAE focuses on smooth probability distributions and effectively averages possibilities, so fine, sharp details can get lost. GANs usually create sharper images. (Modern image generators like Stable Diffusion actually combine ideas: they use a VAE to compress images and a diffusion model to generate inside that compressed space.)
Where VAEs are used
Image generation
Create new images.
Anomaly detection
Data the VAE rebuilds badly is probably unusual.
Drug discovery
Generate new molecular structures.
Recommendation systems
Learn hidden user preferences.
Data compression
Efficient representation learning.
The core idea to remember
- Encoder: “What are the important hidden features?”
- Latent space: “Store those features in an organised, probabilistic form.”
- Decoder: “Use those features to recreate or generate data.”
- GAN learns realism through competition. VAE learns hidden structure through compression and probability.
Check your understanding
1. What does a VAE's encoder output?
The encoder maps input to a region of latent space (a mean and a spread), not a single point.
2. Why can a VAE generate new data but a plain autoencoder usually can't?
Smoothness means there are no “holes”: any nearby point gives something realistic.
3. Compared with GANs, VAE images are usually…
Averaging over possibilities blurs detail, but training is far more stable.