openai.decode

Returns the text from its tokenized representation.

This function doesn’t call the API, decoding is performed locally.

Syntax

decode(encoding, tokens, specialTokens?)

encoding

The encoding to use. Currently available encodings:
- cl100k_base
- p50k_base
- p50k_edit
- r50k_base
- gpt2
You can get the encoding for a model using openai.encodingForModel. For example, gpt-3.5-turbo and gpt-4 models use "cl100k_base".
tokens

An array of integer tokens.
specialTokens optional

An object that maps additional special tokens to their values. For example: { "<|endoftext|>": 50256 }

The decoded string.

Since OpenAI tokenizers don’t have a formal specification, the current decoder may differ from the one used by OpenAI in some cases.

openai.decode("cl100k_base", [15339, 220, 100257])

"hello <|endoftext|>"