Self-supervised Learning

Creating learning targets from data itself, central to language models.

In this lesson

Lesson goals

Masking and prediction
General representations
Transfer to later tasks

Core idea

Creating learning targets from data itself, central to language models.

1. Masking and prediction: understand it in context, identify inputs and outputs, and define how the result will be verified.

2. General representations: understand it in context, identify inputs and outputs, and define how the result will be verified.

3. Transfer to later tasks: understand it in context, identify inputs and outputs, and define how the result will be verified.

Practical example

Consider a real application of “Self-supervised Learning.” Start small, identify available data, and compare the system with a clear baseline before expanding it.

Check your understanding

Explain “Self-supervised Learning” in your own words.
Give one appropriate and one misleading application.
What risk or limitation should be monitored?