Regularization (Ridge + Lasso)
Phase: 1 | Status: ✅ Complete | Prerequisites: 01 Linear Regression, 02 GD
Overview
L2 penalty (Ridge), L1 penalty (Lasso), elastic net, constraint geometry, sparsity, SVD shrinkage, soft-thresholding, Bayesian interpretation (MAP).
Contents
| # | File | Type | Description |
|---|---|---|---|
| 1 | theory.md |
Theory | WHY, objectives, closed forms, SVD shrinkage, subgradients, geometry, Bayesian views |
| 2 | first_principles.ipynb |
Computation | WHY→WHAT→HOW→BUILD→VERIFY — Ridge & Lasso from scratch, sklearn comparison, failure cases |
| 3 | exercises.ipynb |
Practice | Hand derivation, soft-thresholding, sparse recovery, conceptual & failure-analysis questions |
Connections
- Prereqs: 01 Linear Regression, 02 Gradient Descent
- Synthesis: Bias–Variance, Geometry of ML, Regularization Across Models
- Next: 04 Logistic Regression, 09 SVM (regularized margin)