Variational Autoencoders
Variational Autoencoders (VAEs) are generative models that learn a probability distribution over the latent space. Instead of deterministic latent vectors, VAEs learn mean and variance vectors, then sample from a normal distribution. This allows generating new data by sampling from the latent prior.
VAE = probabilistic autoencoder. Encoder outputs parameters of a distribution (μ, σ). Decoder reconstructs from a sampled latent vector.
Key Difference from Standard Autoencoder
Standard autoencoder: latent vector = deterministic output of encoder. VAE: latent vector = sample from N(μ, σ²). This makes the latent space continuous and smooth, enabling interpolation and generation.
Loss Function: Reconstruction + KL Divergence
VAE loss has two terms:
1. Reconstruction loss (how well the output matches input).
2. KL divergence between learned distribution and standard normal prior (regularizes latent space).
The KL term forces the latent distribution to be close to N(0,1), making sampling easier.
Generation: Sample from Prior
After training, discard the encoder. Sample a random vector from N(0,1) and pass it through the decoder to generate new data (e.g., new faces, digits).
# Sampling new digits
z = torch.randn(16, latent_dim)
generated = decoder(z)Applications
- Generating new images (faces, digits, textures).
- Interpolating between data points (e.g., smooth transition between two faces).
- Anomaly detection via reconstruction probability.
Two Minute Drill
- VAEs learn a continuous latent distribution, not a deterministic vector.
- Loss = reconstruction + KL divergence (regularization).
- Generate new data by sampling from N(0,1) and decoding.
- Used for image generation and interpolation.
Need more clarification?
Drop us an email at career@quipoinfotech.com