Seq2Seq
Table of Contents
Of all Neural Network approaches for this problem, a common denominator is the formulation of an encoder and a decoder: resulting in the alias "encoder-decoder networks"
1. Encoder(Seq2Seq)
- maps sequential input to an embedding
- serves as a machine-usable representation of the input sequence
- may be a CNN, RNN, or some other architecture
2. Decoder(Seq2Seq)
- maps the embedding to the desired output sequence
3. Misc
- incorporating Attention modules into the encoder-decoder networks results in better performance
- implemented by an additional set of parameters that combine some information from the encoder and the current state vector of the decoder to generate the label
- results in better context consideration than bidirectional and gated RNN variants.