Token Arcade: Understanding Structured Output and Constrained Decoding Through a Game

01 ABSTRACT

Token Arcade is an interactive game built with Transformers.js that runs locally in the browser via WebGPU. It demonstrates that a language model's top prediction can violate syntactic or format constraints, such as generating a capital letter in a 'heads or tails' task, and then lets users experience constrained decoding by picking cards. The fact is that it uses local inference and gamifies next-token probability and allowedness; the opinion is that it makes structured output easier to understand.

02 KEY FINDINGS

1. The model's top prediction can violate constraints, such as generating a capital letter in 'heads or tails'.
2. Token Arcade lets users experience constrained decoding by picking cards and trying to build a valid JSON creature.
3. It uses Transformers.js and WebGPU to run locally in the browser.