Chat Completion

POST /chat/completion

Creates a model response for chat conversations. OmniAI translates OpenAI Chat Completion params for all models. Supported models can be found here.

Request

Headers

An API Key is required to access this endpoint.

NameValue

x-api-key

your_api_key

Body

NameTypeDescription

messages

List of messages in the OpenAI style.

model

string

The model to use.

max_tokens

number (optional)

The maximum number of tokens to generate before stopping.

stop

string[] (optional)

Stop sequence.

temperature

number (optional)

The randomness of the response from 0 - 2. Higher temperatures generate a more random response. Defaults to 1.

top_p

number (optional)

An alternative to temperature. Defaults to 1.

tools

Tool[] (optional)

A list of tools the model can call.

tool_choice

Tool (optional)

Specifies how the model should use the tools. Forces function calls with each provided function. Defaults to none when no tools are present. Defaults to auto if tools are present.

Message

NameTypeDescription

role

enum

Supported enums: system assistant user

content

string

The contents of the message.

Tool

NameTypeDescription

type

string

The type of tool. Only function is supported.

function

Function object.

Function

NameTypeDescription

description

string

A description of what the function does.

name

string

The name of the function to be called.

parameters

object

The parameters the functions accepts, described as a JSON object.

Example

{
  "model": "meta.llama3-70b-instruct-v1:0",
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant."
    },
    {
      "role": "user",
      "content": "Hello!"
    }
  ]
}

Response

The API returns the response in JSON format.

The request will return a 200 with a chat completion object.

Example

{
  "id": "chatcmpl-123",
  "object": "chat.completion",
  "created": 1677652288,
  "model": "meta.llama3-70b-instruct-v1:0",
  "system_fingerprint": "fp_aa87380ac5",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "Hello! How can I assist you today?",
    },
    "logprobs": null,
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 19,
    "completion_tokens": 9,
    "total_tokens": 28
  }
}

Last updated