Transfer Learning
Table of Contents
1. Basics
- pick an existing model trained on some dataset
- adapt the model to another dataset that is distributionally different from the original one
- for instance, fine tuning a semantic segmentation model trained on roads in Europe, on roads in India.
Elaborating..
1.1. for Neural Networks
- build a deep model on the original dataset
- compile a much smaller labeled dataset for tuning to the final model
- remove some final layers from the first model (should have an embedding layer as final now)
- replace the removed layers with new layers that'll adapt to the new dataset and problem
- "freeze" the parameters of the initial layers of the base first model
- use the smaller labeled dataset to train the parameters of the new final layers (forwards and gradient descent)