decodeSemanticTokens
decodeSemanticTokens(
data,legend,document):SemanticTokenDecodeResult
Turns an LSP semantic-tokens response into absolute spans with names on them.
This is the one decoder. It lives in this package rather than in the editor because it is on the request side of the seam — it runs on the host’s schedule, against the host’s legend, and the editor’s paint layer never sees a legend or a 5-tuple. A second implementation anywhere is a defect: the relative cursor below is stateful, and its rejection rules are the kind that produce plausible wrong offsets rather than exceptions. A copy that drops an out-of-legend tuple without advancing the cursor corrupts every span after it and still paints something.
Five rules, each of which exists because a real server violates it:
- Decode by index. Never invert the legend into a name-to-index map. Real legends ship the
same name at several indices — one server ships
variableat three andfunctionat two — and an inverted map silently mis-decodes every duplicate. - An out-of-legend
tokenTypeIndexdrops the tuple but still advances the cursor. - Modifier bits beyond the legend’s length are ignored, not errors. The bitset is 32 bits wide and a legend may declare six.
- Zero-length tuples are dropped. They cannot paint and they are common in the wild.
- Every offset is clamped to the document length, and a tuple that begins past the end of
the text it addresses —
deltaLinepast the last line, orcharacterpast the end of the line it reached — is dropped rather than throwing. Only the start is held to the line: anendbeyond it is a multi-line span, which is a thing this decoder emits on purpose.
Offsets come out absolute, in UTF-16 code units, which needs no encoding conversion at all: the
client declares general.positionEncodings: ['utf-16'], UTF-16 code units are JavaScript string
indices, and the editor’s paint APIs take offsets.
Parameters
Section titled “Parameters”ArrayLike<number>
legend
Section titled “legend”SemanticTokensLegend