Project Detail
text2token
A simple web app to convert text into byte-level tokens and back. Idea was to understand how could the LLM process the words.
CSSJavaScriptHTML
View on GitHubArchitecture
This repository is reviewed through a modular architecture lens: clear boundaries between UI, domain logic, and external integrations, with emphasis on maintainability and scaling paths.
README
Text2Token
A simple web app to convert text into byte-level tokens and back.
How to use
- Type your text in the box.
- Click Encode to see tokens.
- Click Decode to get the original text.
Run locally
Open index.html in any modern browser.
License
MIT
System Diagram
Replace this reference diagram with project-specific architecture assets when available.
Source: wayiam/text2token