Grammar-constrained generation
Grammar constraints, compiled.
GLRMask is a Rust and Python library for JSON Schema and general context-free grammars. It moves lexer, parser, and tokenizer work ahead of generation, then turns the current parse state into a next-token mask.
The shape of it
Do the awkward work before the GPU needs an answer.
A normal grammar-constrained decoder has to reconcile model tokens with lexer and parser state while decoding. GLRMask deliberately makes compilation heavier so that mask generation has much less structure left to discover online.
Start with JSON Schema
Build a reusable constraint, get a mask from the current state, sample, then commit the chosen model token.
02General CFGs stay general
Lexer ambiguity and multiple parse stacks are represented directly rather than forced into a single early parse.
03Optimize for the hot path
The trade is explicit: more compile work and reusable artifacts in exchange for low, predictable per-token masking latency.
Runtime
The parser is still there. It just isn’t being replayed across the vocabulary.
Committing a sampled token advances the incremental GLR parser. Generating the next mask takes a different path: a deterministic weighted automaton reads the compact parser state and returns the vocabulary mask directly.
MaskRead the current weighted graph-structured stack and produce the legal model-token set.
SampleApply the mask to model logits. The sampler cannot choose a token outside the constraint.
CommitAdvance lexer and GLR parser state only for the token that was actually chosen.
Where to go next
A short path in, then the machinery if you want it.
The site is intentionally layered. You can stop at the practical API, inspect the compilation story, or go all the way into the longer derivations and experiments.
Use GLRMask
Installation, the runtime loop, grammars, serialization, and when to choose compiled versus dynamic constraints.
StoryUnderstand the compiler
Model tokens → terminal sequences → stack effects → Parser DWA → mask. The main idea without a 20-minute detour.
NotebookRead the experiments
Long-form articles, benchmark investigations, lexer edge cases, and the development notes that sit behind the library.
Good fit
Reuse the constraint.
GLRMask makes the most sense when a compiled constraint will serve many requests, or when low and predictable mask latency matters enough to pay compilation up front.
See the trade-offDifferent fit
One schema, one request.
For a brand-new JSON Schema used once, a decoder with very small startup cost can be the better total-latency choice. GLRMask also has a dynamic path for cases where a cached compilation is unavailable.
Crate page