Neural Networks
Phase: 3 | Status: ✅ Complete | Prerequisites: 02 GD, 04 LogReg
Overview
Universal approximation, forward pass, backpropagation, training loop
Contents
| # | File | Type | Description |
|---|---|---|---|
| 1 | theory.md |
Theory | Motivation, notation, forward pass, backprop derivation, initialization, failure cases |
| 2 | first_principles.ipynb |
Computation | MLP from scratch, XOR problem, gradient checking, PyTorch comparison, experiments |
| 3 | exercises.ipynb |
Practice | Hand calculation (forward + backprop), Dense layer gradient check, XOR capacity, vanishing gradients |
Connections
- Prereqs: 02 Gradient Descent, 04 Logistic Regression
- Synthesis: Loss Functions, Optimization Methods
- Next: 14 CNN, 15 RNN/LSTM, 16 Transformer, 17 Autoencoder