Tokens and Context Window
What is Tokens and Context Window?
Tokens are the small pieces of text, often parts of words, that language models read and write. The context window is the maximum number of tokens a model can consider at once, including its instructions, the conversation and any documents supplied.
Tokens matter for two practical reasons. First, cost: most hosted language models are priced per token in and out, so long prompts, long conversations and large documents add up quickly at contact-center volumes. Second, capacity: if a conversation, its history and the supporting knowledge do not fit in the context window, something has to be left out or summarized.
Good AI design therefore sends a model only what it needs for the task: the relevant knowledge passages rather than whole manuals, a summary of earlier contacts rather than every transcript. That keeps answers focused, response times short and running costs predictable, and it is one reason specialized small models are attractive for high-volume work.
Related: Large language model (LLM), Prompt and prompt engineering, Small language model (SLM).