Screenshot of platform.openai.com/tokenizer showing the sentence "Does ChatGPT,
Claude, Anthropic, Llama, Mistral, Gemini, and Perplexity all use Byte-Pair
Encoding (BPE)?" tokenizing to 32 tokens / 105 characters. Visible tabs:
GPT-5.x & O1/3, GPT-4 & GPT-3.5 (legacy), GPT-3 (legacy) — illustrates that
different model generations use different tokenizers.
Co-Authored-By: Claude <noreply@anthropic.com>
Three-panel SVG (input context, LLM black box, predicted next token) with
7-iteration loop generating "The capital of Japan is Tokyo." from the prompt
"What is the capital of Japan?". Includes purple feedback loop showing each
predicted token appended back into the input.
Co-Authored-By: Claude <noreply@anthropic.com>
Shayan Rais