AI token counter and context inspector

See how text, code and JSON use a token budget before adding them to an AI application. Inspect token boundaries, compare two drafts or assemble instructions, history and documents into a context you can measure.

Why a word count is not a token count

A tokenizer splits text into pieces from a fixed vocabulary. A common word may use one token; punctuation, spacing, source code and non-English text can split differently. Counting characters and dividing by four cannot give an exact result for arbitrary input.

This counter uses o200k_base and cl100k_base, two named BPE encodings, through the bundled gpt-tokenizer library. Select the encoding appropriate to your application. Counts are exact for the chosen encoding and the literal text supplied; they are not universal counts for every AI model. No AI model runs, and no input is sent to an AI service. Tokenization runs on your device in a background browser thread, keeping the page responsive.

The preview shows token boundaries and can reveal spaces, line breaks and token IDs. Some tokens contain only part of a UTF-8 character. Those tokens are grouped visually so emoji and other characters remain readable; their individual IDs are retained. The character metric counts Unicode code points, not necessarily visible symbols: a joined emoji can contain several code points.

Which tokenizer encoding should I choose?

An encoding defines the vocabulary and rules used to split text into token IDs. Choose the one your target model uses. The 100k and 200k names refer to vocabulary families, not the model’s context capacity.

For example, お誕生日おめでとう uses 8 tokens with o200k_base and 9 with cl100k_base. Two encodings can also return the same count while assigning different pieces or IDs. Switching the selector recounts your text; it does not change your model or reduce that model’s bill.

These examples follow OpenAI’s tokenizer guide . Check the exact model/version you use rather than assuming every newer model uses the same encoding. Neither option represents Claude, Gemini or every other provider. If you are just exploring, keep the default and switch encodings to see the differences.

Compare a prompt or plan its context

Use Compare text to measure an original and a revised version. A smaller count tells you that the selected encoding uses fewer tokens, not that the new prompt will produce a better answer.

In Plan context, fill the blocks you need. Nonempty blocks are joined in their displayed order with two line breaks. The assembled text is counted again rather than adding the separate block counts. Separators and changed token boundaries can affect the result. The tool does not turn these blocks into API messages or add role names.

Enter a context capacity, an output reserve and any extra overhead allowance. The defaults are an illustrative budget, not a model configuration. The remaining space is:

remaining = capacity - text tokens - output reserve - extra overhead

API message formatting, tool definitions as processed by a provider, images and other non-text inputs may contribute additional tokens. Check your provider’s requirements separately. A text-only budget cannot guarantee that a complete request fits.

For the request itself, the cURL command builder can prepare a command without sending it. Our JSON, form and multipart request guide explains why those body formats are not interchangeable. Text is not transferred between the tools.

Estimate a cost scenario with your own rates

Open Cost scenario and enter rates in USD per million tokens. Supply the expected output, the number of repeated calls and the portion of input you expect to be billed at the cached rate. The calculation uses the inspected input plus your overhead allowance. In comparison mode, it uses version A.

cost per call = ((input - cached) × input rate
                + cached × cached rate
                + expected output × output rate) / 1,000,000

The expected output is separate from the reserved space. The starting rates are $0.75 input, $0.075 cached input and $4.50 output per million tokens, based on GPT-5.4 mini standard text pricing , checked October 27, 2026. They are editable examples, not a model selection or a live price feed. Changing the tokenizer leaves your rates unchanged. Cache eligibility, reasoning usage, tool charges, taxes and other fees are not inferred. The estimate assumes the same input and output on each call; a conversation that grows between calls needs a different scenario.

Keep a reviewable result

Copy or download the JSON report to retain the input, selected encoding, full token IDs and valid budget/cost settings. It includes your text, so review it before sharing. User text is not saved in browser storage or share URLs; working text and results are cleared when you leave the page. Ordinary page and asset requests still occur, as explained in the browser Network inspection guide.

The active inputs are limited to 50,000 UTF-16 units, including separators in context mode. Many emoji occupy two units. The visualization shows approximately the first 600 tokens, extending a group if needed to finish a character; counts and report IDs cover the full accepted text. Incomplete Unicode is rejected, and a long-running analysis can be cancelled. Special-token spellings such as <|endoftext|> are counted as ordinary text.

Learn more

• OpenAI: counting tokens  explains why full API requests include more than locally tokenized text. Its counting API is not used by this tool.
• Claude: token counting  describes provider-specific request counting. Neither encoding here is presented as Claude’s tokenizer.
• GPT Tokenizer playground  provides another way to explore the library’s tokenization behavior.