Meta Learning

What’s meta learning?

Meta means referring to oneself. Meta learning means learning, for learning. Or learning in order to learn better, or so on. Meta learning focuses on help models learn faster on the first run, training faster without needing to worry about bad things. Learning new things fast and efficient — That’s the realm of meta learning.

Difference with Life Long Learning.

Life long learning refers to a model learning and adapting to different environments and datasets through out the lifespan of the model (not just when it’s trained). So, how’s that different from meta learning?Life long learning focuses on the improvement of new tasks with the same model, in contrast, meta learning hopes to learn a good initial model that can learn many tasks fast.

How to meta learning?

In traditional ML methods, we are given a lot of pairs of input and target data. To train a model, we initialize the model with some parameters, then update the parameters towards the goal of minimization of the loss function between the target and the model’s output.

In meta learning however, instead of finding the global minimum right off the bat, we are searching for a location from which it’s easy to find global minimum for each tasks. In other words, we are pre-training a model such that each sub-task can be trained really fast. For example, in MAML and Reptile, two meta learning algorithms, models are updated towards minimizing each task, not just one single task in particular.