C1Linear & GLMs

Linear models and generalized linear models are the workhorses of classical machine learning: a weighted sum of features, optionally squeezed through a link function, that turns inputs into a prediction. They are fast to fit, easy to interpret, and backed by clean theory, which is why they remain the default first model on most tabular problems and the baseline everything fancier has to beat.

You will derive ordinary least squares from the normal equations and read it as a projection onto the column space, then see the same fit fall out of maximum likelihood under Gaussian noise. From there you will work through the bias-variance tradeoff, the ridge, LASSO, and elastic-net penalties (and why L1 yields sparse coefficients), logistic and softmax regression for classification, and the exponential-family framework that unifies them with Poisson and Gamma GLMs for counts and durations.

These ideas recur everywhere downstream: a neural network is a stack of linear models with nonlinearities between them, regularization shows up in every training loop, and the cross-entropy loss you meet here is the loss that trains modern language models.