E2Pre-Transformer neural NLP

This discipline covers the neural NLP that came after counting but before the Transformer: the era when models learned to represent words as dense vectors and to read sequences with recurrence. It solves the core weakness of n-grams, the inability to generalize across similar words, by placing meaning in a continuous space where 'king' and 'queen' sit close together.

You will train word2vec with skip-gram negative sampling, derive GloVe from co-occurrence ratios, and add fastText's subword composition for rare and out-of-vocabulary words. From static embeddings you move to recurrent language models, deriving backpropagation through time and the vanishing-gradient problem, then defeat it with LSTM and GRU gates. You finish with ELMo's contextual embeddings and a seq2seq encoder-decoder equipped with Bahdanau attention.

This is the bridge to everything that follows: attention is introduced here in its original additive form, and the encoder-decoder shape you build is the direct ancestor of the Transformer in E3.