Type Alias LLMOptions

LLMOptions: {
    completion_first?: boolean | null;
    frequency_penalty?: number | null;
    image_config?: ImageConfig | null;
    max_tokens?: number | null;
    presence_penalty?: number | null;
    stop_tokens?: string[] | null;
    stream_response?: boolean | null;
    system_prompt?: string | null;
    temperature?: number | null;
}

LLM options to use for the completion. If not specified, this defaults to the dataset's LLM options.

Type declaration

  • Optionalcompletion_first?: boolean | null

    Completion first decides whether the stream should contain the stream of the completion response or the chunks first. Default is false. Keep in mind that || is used to separate the chunks from the completion response. If || is in the completion then you may want to split on ||{ instead.

  • Optionalfrequency_penalty?: number | null

    Frequency penalty is a number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim. Default is 0.7.

  • Optionalimage_config?: ImageConfig | null
  • Optionalmax_tokens?: number | null

    The maximum number of tokens to generate in the chat completion. Default is None.

  • Optionalpresence_penalty?: number | null

    Presence penalty is a number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics. Default is 0.7.

  • Optionalstop_tokens?: string[] | null

    Stop tokens are up to 4 sequences where the API will stop generating further tokens. Default is None.

  • Optionalstream_response?: boolean | null

    Whether or not to stream the response. If this is set to true or not included, the response will be a stream. If this is set to false, the response will be a normal JSON response. Default is true.

  • Optionalsystem_prompt?: string | null

    Optionally, override the system prompt in dataset server settings.

  • Optionaltemperature?: number | null

    What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. Default is 0.5.

Settings

Member Visibility