AI Token Counter for GPT, Claude and Gemini

What a token actually is

Language models do not read characters or words. They read tokens: chunks of text produced by a tokenizer, typically three to four characters long in English. The word tokenization might be one token; the word antidisestablishmentarianism is several. Common words are usually a single token, and the leading space is generally part of it, which is why " the" and "the" can tokenize differently.

Tokens matter for three practical reasons. They are the billing unit, so cost is proportional to them rather than to words. They are the context limit, so a document either fits in the window or it does not. And they govern latency, because generation time scales with the number of tokens produced.

A rough rule for English prose: one token is about four characters, or roughly 0.75 words. A thousand tokens is about 750 words, or a page and a half. Those ratios shift considerably for code, for languages that do not use Latin script, and for text dense with numbers or punctuation — which is exactly why an estimator that accounts for content type is more useful than multiplying by a constant.

How accurate this estimate is

This tool estimates rather than tokenizes exactly, and it is worth being clear about what that means.

Running a real BPE tokenizer in the browser requires downloading a vocabulary file of several megabytes per model family, and those vocabularies change between model versions. Instead, this counter analyses the text — the balance of words, punctuation, digits, whitespace and non-Latin characters — and applies ratios calibrated to how each family's tokenizer behaves on that kind of content. For ordinary English prose the result typically lands within five per cent of the true count. For source code, dense JSON or heavily mixed-script text, expect closer to ten.

That is accurate enough for the decisions people actually make with a token counter: will this document fit, roughly what will this cost, is this prompt twice as long as it needs to be. It is not accurate enough to reconcile an invoice. Every provider returns exact input and output token counts in the API response, and that figure is the one to trust for billing. Treat this as a planning instrument, and leave a margin — if your estimate says 95% of the context window, assume it will not fit.

What makes token counts higher than you expect

Several kinds of content tokenize far less efficiently than plain English, and they are common in real prompts:

• JSON and code. Braces, quotes, colons and indentation each consume tokens. A JSON payload can use two to three times the tokens of the same information written as prose — which is a real argument for compact formats when the model does not need the structure.
• Non-Latin scripts. Chinese, Japanese, Arabic, Hindi and Urdu tokenize much less efficiently than English, sometimes at one token per character. The same document translated can cost several times as much to process.
• Long numbers and identifiers. UUIDs, hashes and long numeric strings fragment heavily, because they do not appear in the tokenizer's vocabulary as units.
• Unusual formatting. Repeated whitespace, ASCII tables and heavy markdown all add tokens that carry very little meaning.
• The system prompt and chat scaffolding. Every message in a conversation carries a few tokens of structural overhead, and the whole history is resent on each turn — which is why a long conversation gets progressively more expensive per reply.

Once you can see the count, reducing it usually becomes obvious: strip the boilerplate, send the fields the model needs rather than the whole record, and summarise earlier turns instead of resending them verbatim.

How to use the AI Token Counter

1. Paste your text or prompt

   Anything works — a system prompt, a document you plan to summarise, a code file, or a whole conversation transcript.

2. Read the token estimate

   The large figure is the estimate for the selected model family. Characters, words and lines are shown alongside it for reference.

3. Check the context window bars

   Each bar shows what share of that model's window your text occupies. Remember to leave room for the response, which comes out of the same budget on most models.

4. Estimate the cost

   Take the token count to the LLM API cost calculator to turn it into a figure per call and per month.

Frequently asked questions