You can use AI models via AI cells or via ai module in Code cells.
The default model is set in Settings ▸ AI. Each codebook and the scratchpad has a separate model configuration, which is initially set to the default model but can be changed in the Codebook settings.
The Temperature parameter controls the randomness of the model. A higher temperature, such as 0.8 or 1.0, tends to produce more creative but potentially less coherent or focused responses, while a lower temperature, such as 0.2 or 0.5, tends to produce more more deterministic and focused outputs. The default temperature is 0.5.
You can download and manage local AI models in Settings ▸ AI. These models work offline on your computer.
Local models require significant amount of memory (RAM) to operate. If there isn’t enough free memory available, they may run slowly. To ensure that they work as intended, you may need to quit other applications.
Model name in code: llama2-7b-chat.
Llama 2 7B-Chat is a 7-billion parameter transformer model for text generation, fine-tuned for chat.
Quantization: 4-bit.
Created by Meta. Quantized by Tom Jobbins.
License: Llama 2 Community License Agreement.
Model name in code: openchat.
OpenChat is a 7-billion parameter transformer model for text generation, fine-tuned for chat.
Version: OpenChat-3.5-1210.
Based on the Mistral 7B model created by Mistral AI, fine-tuned by Wang, Guan and Cheng, Sijie and Zhan, Xianyuan and Li, Xiangang and Song, Sen and Liu, Yang. Quantized by Tom Jobbins.
Quantization: 4-bit.
License: Apache 2.0
Model name in code: neural-chat-7b.
Neural Chat is a 7-billion parameter transformer model for text generation, fine-tuned for chat.
Based on the Mistral 7B model created by Mistral AI, fine-tuned by Intel.
Quantization: 4-bit.
License: Apache 2.0
Model name in code: tinyllama-chat.
TinyLlama-Chat is a chat-tuned 1.1B-parameter transformer model for text generation.
Created by Peiyuan Zhang, Guangtao Zeng, Tianduo Wang and Wei Lu.
Quantization: 5-bit quantized.
License: Apache 2.0
Remote models are called via the corresponding APIs.
In order for them to work, you must supply a correct API key in Settings ▸ Keys.
The full OpenAI API is also available via openai module.
Model name in code: gpt-3.5-turbo.
The original ChatGPT model.
Model name in code: gpt-4-turbo.
The GPT-4 turbo model.
Model name in code: gpt-4.
The GPT-4 model.
Model name in code: command.
The Cohere Command model.
Model name in code: command-light.
The Cohere Command-light model.
Model name in code: textsynth-gptj.
GPT-J 6B model using TextSynth API.
Model name in code: textsynth-gpt-neox-20b.
GPT-NeoX 20B model using TextSynth API.