A grammar-constrained model in your browser
TinyStories-1M runs locally; glrmask WebAssembly applies an exact JSON mask to its BPE logits before sampling.
Live browser demo
Generate a JSON object locally.
TinyStories-1M runs in this tab. Before every constrained sample, a small glrmask WebAssembly runtime returns the exact bitset of legal TinyStories BPE tokens for a precompiled JSON constraint.
- Model
- TinyStories-1M · q4
- Runtime
- glrmask WASM · 1.4 MB
- Constraint
- TinyStories BPE · 1.3 MB
Nothing has been downloaded. Loading the model is explicit; browser inference begins only after that.
Generated BPE pieces appear separately. Hovering a piece keeps that sampling step selected; click it to fix the ledger there, then click it again to unfix it.
- Grammar filter
- not loaded
- Legal BPE tokens
- —
- Selected piece
- —
Selected sampling step
What the model could have emitted.
These are TinyStories’ highest-scoring next BPE pieces at one step. In exact mode, the grammar removes the hatched pieces before sampling.
The model files are fetched from the public TinyStories ONNX repository only after you load them. Generation and masking run in the browser. Prompt and schema presets fill editable fields. The compiled constraint is specific to this tokenizer vocabulary; it does not compile an arbitrary schema in the page. Exact mode remains tied to the bundled Story only preset.
The panel above runs a real, very small language model in the browser. It downloads TinyStories-1M in q4 form after you press the load button, then generates a short continuation from the supplied context.
The constrained mode does more than check the resulting text after the fact. Before each sample, the model has a logit for every BPE token in its vocabulary. The glrmask WebAssembly runtime returns a bitset over that same vocabulary. Tokens whose bits are clear receive (-\infty) before sampling. The selected BPE token is then committed back into the runtime, which advances the JSON constraint for the next step.
The demo keeps the compiled constraint deliberately narrow. It was built for TinyStories’ GPT-2-style BPE vocabulary and one JSON output language. That makes the browser side small: the runtime is about 1.4 MB and the already-compiled constraint about 1.3 MB. The model is still fetched separately, because it is the expensive part and should not arrive until someone asks for it.
What the page is showing
Each generated BPE piece carries the model decision that produced it. Hover a piece in the continuation to inspect that step; that selection stays in place until another piece is inspected. Click a piece to fix the ledger there while you compare candidates. The ledger always contains eleven rows: the sampled piece plus ten candidates. In exact mode it reserves at least two legal and two removed pieces when those classes exist, then fills every remaining slot by the real pre-filter ranking, so the split changes with the step rather than staying artificially even. A forced JSON punctuation step can genuinely have no other legal BPE piece. The displayed percentages are model probabilities computed from the pre-filter logits. The colour of each generated piece is also derived from its sampled logit: redder means the model rated it relatively poorly, greener means it rated it highly.
The prompt and JSON Schema selectors offer a few useful starting points, while both fields remain editable. A hand edit switches the relevant selector to Custom. The selected schema is always included in TinyStories’ instruction, including in raw mode. The downloaded glrmask artifact is deliberately narrower: it is compiled for the Story only preset, so the other schema presets and custom schemas run in raw mode. Choosing Story only restores exact mode. There is no arbitrary-schema compiler in the page.
The output is replayed at a fixed tokens-per-second rate so the table has time to be read. In the development control panel, Limit output TPS can set that rate or turn the cap off. The model may finish its underlying computation earlier; the cap governs the visible token-by-token trace and the corresponding ledger updates. Stop interrupts the generation loop at its next decoding boundary and discards any queued-but-not-yet-visible pieces.
The model is tiny, so this is a systems demonstration rather than an attempt at high-quality JSON prose. The useful part is the placement of the mask: the model and the constraint live in the same browser process, and the mask is applied before each random choice.
The runtime loop
For each generated token, the page performs this sequence:
- Reads the mask words from the WebAssembly session.
- Applies that exact bitset to the TinyStories logits.
- Samples from the remaining BPE tokens.
- Commits the sampled token id back to the WebAssembly session.
- Decodes the sampled id for display.
The runtime never receives an already-decoded string to validate. It advances on token ids, so the tokenizer vocabulary used by the model and the vocabulary used by the constraint have to match exactly. The compiled artifact bundled with this page is specific to TinyStories for that reason.
Scope
This page is intentionally a small deployment target. It demonstrates that a compiled grammar constraint can travel as a browser asset alongside a small model. It does not claim that a 1M-parameter story model is generally useful, or that every schema can be compiled cheaply in a tab. The rest of the project is about making those compilation and masking costs tractable for much larger grammars and vocabularies.