Autoencoder
Phase: 3 | Status: ✅ Complete | Prerequisites: 13 Neural Networks, 10 PCA, Information Theory
Overview
Encoder-decoder architecture, representation learning via reconstruction, linear autoencoder–PCA equivalence, denoising and sparse variants, variational autoencoder (ELBO, reparameterization trick, KL regularization).
Contents
| # | File | Type | Description |
|---|---|---|---|
| 1 | theory.md |
Theory | Architecture, PCA connection, denoising/sparse/variational AE, ELBO derivation, reparameterization trick, failure cases |
| 2 | first_principles.ipynb |
Computation | Linear AE ↔ PCA, nonlinear AE, denoising, VAE from scratch, latent space visualization, PyTorch comparison |
| 3 | exercises.ipynb |
Practice | Hand calculation (linear AE reconstruction), reparameterization trick coding, KL divergence conceptual analysis |
Connections
- Prereqs: 13 Neural Networks, 10 PCA, Information Theory
- Synthesis: Probabilistic View of ML, Geometry of ML
- Next: Generative models (GANs, diffusion)