Count text tokens locally with o200k_base or cl100k_base. Inspect token pieces, token IDs, and UTF-8 byte counts in your browser.

Ready to process.

Processed in this browser. No uploads or automatic saving.

Text tokens
-
Unicode code points
-
UTF-8 bytes
-

Token pieces and IDs

Showing up to 500 tokens; the total covers the entire input. Replacement characters can occur at UTF-8 byte boundaries between tokens.

When should I use an AI token counter?

Count prompts, source code, JSON, or multilingual material before planning context budgets or estimating API input costs. Select an encoding and calculate to see tokens, Unicode code points, UTF-8 bytes, and the first 500 token pieces and IDs. Tokenization happens in your browser.

How do o200k_base and cl100k_base differ?

These BPE encodings have different vocabularies, so identical prose, Chinese text, or source code can produce different counts. Choose the encoding documented for your target model. This tool identifies the encoding rather than claiming one universal tokenizer for Claude, Gemini, DeepSeek, or models whose encoding has not been verified.

Why is one word or Chinese character not always one token?

A token can represent a word, part of a word, punctuation, or bytes. Chinese, emoji, whitespace, and code symbols split according to the vocabulary. A fixed characters-per-token ratio is only an approximation. This tool executes the selected tokenizer. An individual token spanning part of a UTF-8 character may display a replacement character without altering the original text.

Is this the final API billing count?

No. This tool counts raw text and treats special markers as ordinary text. It does not add chat roles, tools, system wrappers, image or audio input, or predict reply and reasoning tokens. Use provider usage data for billing. The input limit is 100,000 characters and processing runs in a browser worker.

References: gpt-tokenizer