Transfer Learning vs Domain Adaptation

How is TL different from DA?

In the previous section, we have introduced TL and DA as if they are the same thing. While there are some arguments whether DA is a part of TL, we lean towards treating them as cousins, similar, but different.

Transfer learning definition.

Transfer learning focuses on storing knowledge gained while solving one problem and applying it to a different but related problem. For example, knowledge gained while learning to recognize cars could apply when trying to recognize trucks.

– Wikipedia

Domain adaptation definition.

Domain adaptation is used when we aim at learning from a source data distribution, a well performing model on a different (but related) target data distribution. For example, an algorithm trained on news-wires might have to adapt to a new dataset of biomedical documents.

– Wikipedia

In simple terms.

See the difference? Transfer learning is used when you want to transfer the knowledge of a model over to another model, while domain adaptation is used when you want to adapt a model to what it has never seen before. Transfer learning is about keeping what a model knows, and domain adaptation is more about making a model work in a new environment.

In other words, in DA the input distribution changes but the labels remain the same; in TL, the input distributions stays the same, but the labels change.