Regularization
Regularization adds a penalty to the loss function to discourage model complexity, reducing overfitting. Common techniques: Ridge (L2) and Lasso (L1).
Regularization shrinks coefficients or forces some to zero, simplifying the model.
Ridge Regression (L2)
Adds penalty equal to sum of squared coefficients. Shrinks coefficients toward zero but never exactly zero. Good when many features have small effects.
from sklearn.linear_model import Ridge
ridge = Ridge(alpha=1.0) # alpha = regularization strengthLasso Regression (L1)
Adds penalty equal to sum of absolute coefficients. Can shrink coefficients to exactly zero – performs feature selection.
from sklearn.linear_model import Lasso
lasso = Lasso(alpha=1.0)Elastic Net (L1 + L2)
Combines both penalties. Best when there are correlated features.
from sklearn.linear_model import ElasticNet
elastic = ElasticNet(alpha=1.0, l1_ratio=0.5)Regularization in Other Models
- Decision trees: limit max_depth, min_samples_split.
- Neural networks: dropout, weight decay.
- SVM: C parameter (smaller C = more regularization).
Two Minute Drill
- Regularization reduces overfitting by penalizing complexity.
- Ridge (L2) shrinks coefficients.
- Lasso (L1) can zero out coefficients (feature selection).
- Alpha controls strength – larger alpha = more regularization.
Need more clarification?
Drop us an email at career@quipoinfotech.com