AI Glossary
What is a token in AI?
Token: A token is the small chunk of text — roughly three-quarters of an English word — that an AI model reads and writes, one at a time.
In plain English
Models don't see words or letters; they see tokens. "PromptAI publishes daily" is about 5 tokens. Common words are usually one token; rare words get split into pieces. Non-English scripts like Devanagari often cost more tokens per word, which is why the same sentence in Hindi can "use up" more of a model's capacity than in English.
Why it matters
- AI pricing is per token — both what you send and what it replies
- Context limits ("128k context") are measured in tokens, not pages
- Long documents get cut off when they exceed the token budget
A concrete example
The word "unbelievable" might be split into "un", "believ", "able" — three tokens. The model predicts each piece in sequence.
Related terms
Learning AI from scratch? Start with the free Learn AI course picks, then put it to work with 50 tested prompts — or get the daily 2-minute briefing, free →