Your files stay on your device
OpenAI tokenizer playground
Count tokens exactly with the encoding a model actually uses. Every token id and slice listed, both directions, computed in a worker.
no Web Workers in this environment: exact counts need them; the page stays honest instead
Paste text (or ids), then press Count.
The same text through o200k_base, cl100k_base, p50k_base, and r50k_base. Exact BPE counts per row. The spread between vocabularies is a fact, not a ranking.
- Counts are the BPE output of the installed gpt-tokenizer encoding, a port of OpenAI's tiktoken. Exact for these encodings: what the matching OpenAI models charge for is what is counted.
- Claude, Gemini, and other vendors use different tokenizers. Counts here are NOT their counts. The context-fit tool labels those as estimates instead of pretending.
- Special tokens pasted into the text (like <|endoftext|>) count as their single special id: that is what a model actually receives.
- A token can hold partial UTF-8 (an emoji often splits); its slice may show the replacement character. That is the real boundary, not a bug.
Ordinary English: watch spaces lead most tokens.
Astral emoji and CJK carry multi-byte tokens; one grapheme is not one token.
Code tokens split on punctuation runs; why code costs more tokens than prose.
A pasted special token counts as its single special id; what the model actually sees.
Paste, drop, or type to begin. Everything stays on this device.