Chat Completion

POST /chat/completion

Creates a model response for chat conversations. OmniAI translates OpenAI Chat Completion params for all models. Supported models can be found here.

Request

Headers

An API Key is required to access this endpoint.

Name	Value
x-api-key	your_api_key

Body

Name	Type	Description
messages	Message[]	List of messages in the OpenAI style.
model	string	The model to use.
max_tokens	number (optional)	The maximum number of tokens to generate before stopping.
stop	string[] (optional)	Stop sequence.
temperature	number (optional)	The randomness of the response from 0 - 2. Higher temperatures generate a more random response. Defaults to 1.
top_p	number (optional)	An alternative to temperature. Defaults to 1.
tools	Tool[] (optional)	A list of tools the model can call.
tool_choice	Tool (optional)	Specifies how the model should use the tools. Forces function calls with each provided function. Defaults to none when no tools are present. Defaults to auto if tools are present.

Message

Name	Type	Description
role	enum	Supported enums: system assistant user
content	string	The contents of the message.

Tool

Name	Type	Description
type	string	The type of tool. Only function is supported.
function	Function	Function object.

Function

Name	Type	Description
description	string	A description of what the function does.
name	string	The name of the function to be called.
parameters	object	The parameters the functions accepts, described as a JSON object.

Example

{
  "model": "meta.llama3-70b-instruct-v1:0",
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant."
    },
    {
      "role": "user",
      "content": "Hello!"
    }
  ]
}

Response

The API returns the response in JSON format.

The request will return a 200 with a chat completion object.

Example

200

{
  "id": "chatcmpl-123",
  "object": "chat.completion",
  "created": 1677652288,
  "model": "meta.llama3-70b-instruct-v1:0",
  "system_fingerprint": "fp_aa87380ac5",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "Hello! How can I assist you today?",
    },
    "logprobs": null,
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 19,
    "completion_tokens": 9,
    "total_tokens": 28
  }
}

400

{
  "error": "Invalid request"
}