Tokens and Context Windows

Lesson goals

• A token is not always a word
• Context is limited
• Information order matters

Core idea

How models split input and what they can retain in a session.

1. A token is not always a word: understand it in context, identify inputs and outputs, and define how the result will be verified.

2. Context is limited: understand it in context, identify inputs and outputs, and define how the result will be verified.

3. Information order matters: understand it in context, identify inputs and outputs, and define how the result will be verified.

Practical example

Consider a real application of “Tokens and Context Windows.” Start small, identify available data, and compare the system with a clear baseline before expanding it.

Check your understanding

1. Explain “Tokens and Context Windows” in your own words.
2. Give one appropriate and one misleading application.
3. What risk or limitation should be monitored?