Cost
Hosted models bill by token, but not every token costs the same. Output usually costs more than input. Cached input can be much cheaper. Some models charge a higher rate once a prompt passes a size threshold.
Map · Foundations · 01 · Tokens
Concept card · Foundations
Tokens
Tokens are the pieces of text a model reads, writes, and bills for.
The unit models read, write, and bill in.
What changed
Oct 16: Cost now covers output versus input, cached input, and a higher rate past a size threshold, in general terms. The common mistake is that the same text is not the same token count on every model.
1 What it is
Models don't see words. They see tokens: chunks of text, often parts of words. Every prompt and every reply is counted in tokens.
2 Why it matters
Quality
Tokens spent on leftover instructions are tokens the reply cannot use.
Risk
Anything you paste is sent as tokens, including keys, customer text, and private notes.
3 How to apply · try today
Paste one real prompt into your provider's token counter. Note the input and output tokens, then multiply by the current rates. That gives you the cost per run.
Common mistake
Assuming the same text is the same number of tokens on every model. Tokenizers change between model versions.
Self-check
A prompt is longer than a model's size threshold. What can happen to the input rate?
Coming soon. One pass over a prompt you already send, cutting text that does not change the task.
Coming soon. How a real tokenizer differs from this slice illustration.
Questions
- What is a token?
- Tokens are the pieces of text a model reads, writes, and bills for. A token is often part of a word, not the whole word.
- Is the same text the same number of tokens on every model?
- No. Tokenizers change between models and between versions of a model.
- What can happen to the input rate past a size threshold?
- Some models charge a higher rate once a prompt passes a size threshold.