Note: Due to the decentralized nature of the Lilypad Network we recommend using the streaming variant where possible at this time
This endpoint provides both a streaming interface using Server-Sent Events (SSE) and non-streaming interface for chat completions which is compliant with the OpenAI specification. This means that you can plug and play Anura using the OpenAI SDK by simply passing in the Anura Url and API Key into your client like so:
import OpenAI from 'openai';
const client = new OpenAI({
baseURL: 'https://anura-testnet.lilypad.tech/api/v1',
apiKey: process.env.ANURA_API_KEY || '',
});
const completion = await client.chat.completions.create({
model: 'llama3.1:8b',
messages: [
{ role: 'system', content: 'You are a helpful AI assistant.' },
{ role: 'user', content: 'Are semicolons optional in JavaScript?' },
],
});
return completion.choices[0].message.content;
Request Headers
Content-Type: application/json*
Accept: text/event-stream (recommended for streaming)
Authorization: Bearer YOUR_API_KEY*
Request Parameters
Parameter
Description
Type
model*
Model ID used to generate the response (e.g. deepseek-r1:7b). Required.
string
messages*
A list of messages comprising the conversation so far. Required.
array
Optional Parameters and Default Values
Paraneter
Description
Default
frequency_penalty
Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.
0
max_tokens
The maximum number of tokens that can be generated in the chat completion.
presence_penalty
Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.
0
response_format
seed
If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. Determinism is not guaranteed, and you should refer to the system_fingerprint response parameter to monitor changes in the backend.
null
stop
Up to 4 sequences where the API will stop generating further tokens. The returned text will not contain the stop sequence.
stream
false
stream_options
Options for streaming response. Only set this when you set stream: true.
null
temperature
What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. We generally recommend altering this or top_p but not both.
1
tools
top_p
An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.
We generally recommend altering this or temperature but not both.
1
Request Body (non-streaming)
{
"model": "llama3.1:8b",
"messages": [
{
"role": "system",
"content": "you are a helpful AI assistant"
},
{
"role": "user",
"content": "write a haiku about lilypads"
}
],
"temperature": 0.6
}
Response Format (non-streaming)
The response is an OpenAI ChatCompletion Object with the following format:
An object specifying the format that the model must output. .
If set to true, the model response data will be streamed to the client as it is generated using .
A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for. A max of 128 functions are supported. .
