Classical Machine Learning

Classical machine learning is everything that turns features into predictions without a deep network, and for most tabular problems it is still the right tool: strong theory, modest data requirements, and models you can actually explain. Long before transformers, these algorithms were the field, and gradient-boosted trees remain the quiet winner of a large share of real-world prediction tasks.

You will cover linear models and GLMs, margin and kernel methods (support vector machines), tree-based methods and gradient boosting, instance-based and probabilistic classifiers, and unsupervised learning (clustering and dimensionality reduction). On top of that sit anomaly detection, ensembles, recommendation systems, and approximate nearest neighbor search with vector databases, the retrieval substrate that modern LLM applications also lean on. The pillar closes with how to evaluate and select among all of these without fooling yourself.

Master this pillar and you will reach for a 200-line scikit-learn pipeline before a GPU, and be right to.