NLP & Language Models

This pillar traces the path from counting words to models that write code, hold conversations, and reason through multi-step problems. It is the densest frontier track in the tree, and the one moving fastest, so the emphasis is on the durable ideas underneath the headline models.

You will begin with classical NLP (tokenization, n-gram language models, and parsing), then pre-Transformer neural NLP (word embeddings and recurrent encoders), before reaching the Transformer and modern language models (attention, positional encoding, and mixture-of-experts). From there the pillar covers training LLMs (scaling laws, pretraining, alignment, and parameter-efficient fine-tuning), inference and serving (KV caching, batching, and engines like vLLM), and LLM agents and retrieval (RAG, tool use, and test-time compute). It closes with multimodal models and with interpretability and alignment science, the study of what these systems are actually doing and how to keep them steerable.

If pillar D taught you how to train a network, this pillar is where that skill meets the models reshaping the field.