Generate Completions

Generate text completions with streaming support.

This endpoint generates text completions using specified language models. Supports both streaming (real-time response chunks) and non-streaming modes. Ideal for text generation, continuation, and completion tasks.

Request Body:

• model: Model ID for text generation (required)
• prompt: Input to complete (required) - Can be:
  • Single string
  • List of strings
  • Array of integers (tokenized prompt)
  • Array of integer arrays (batch of tokenized prompts)
• max_tokens (optional): Maximum tokens to generate (default: 16, minimum: 0)
• temperature (optional): Sampling temperature between 0 and 2 (default: 1). Higher values like 0.8 make output more random, lower values like 0.2 make it more focused and deterministic
• stream (optional): Enable streaming responses (default: false)

Optional Headers:

• X-Session-ID: Session identifier for conversation tracking

Returns:

Non-streaming mode -

• id: Unique identifier for the response
• object: Object type (always text_completion)
• created: Unix timestamp when response was generated
• model: Model ID used for completion
• choices: List of completion choices, each containing:
  • text: Generated completion text
  • index: Index of this choice
  • finish_reason: Reason model stopped (stop for natural stop point, length if max tokens reached)
• usage: Token usage statistics:
  • prompt_tokens: Number of tokens in the prompt
  • completion_tokens: Number of tokens generated
  • total_tokens: Total tokens used (prompt + completion)
• confidence_score (optional): Confidence score from Domain-Aligned AI models

Streaming mode - Server-sent events (SSE) stream with StreamingCompletionResponsev2 chunks containing:

• id: Response identifier
• object: Always text_completion
• created: Timestamp
• model: Model ID
• choices: Completion chunks
• usage (optional): Only in final chunk
• confidence_score (optional): Only in final chunk

Example Request (Non-streaming):

POST /api/v3/inference/completions
Headers: {"Authorization": "Bearer <api_key>"}

{
  "model": "nugen-flash-instruct",
  "prompt": "Write a haiku about programming:",
  "max_tokens": 100,
  "temperature": 0.7
}

Example Response (Non-streaming):

{
  "id": "nugen-abc123",
  "object": "text_completion",
  "created": 1704123600.0,
  "model": "nugen-flash-instruct",
  "choices": [
    {
      "text": "Code flows like water,
Bugs emerge, then disappear,
Debug and refine.",
      "index": 0,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 7,
    "completion_tokens": 18,
    "total_tokens": 25
  },
  "confidence_score": 86.5221
}

Example Request (Streaming):

POST /api/v3/inference/completions
Headers: {"Authorization": "Bearer <api_key>"}

{
  "model": "nugen-flash-instruct",
  "prompt": "Write a haiku about programming:",
  "max_tokens": 100,
  "temperature": 0.7,
  "stream": true
}

Example Response (Streaming):

data: {"id":"nugen-abc123","object":"text_completion","created":1704123600.0,"model":"nugen-flash-instruct","choices":[{"text":"Code","index":0,"finish_reason":null}]}

data: {"id":"nugen-abc123","object":"text_completion","created":1704123600.0,"model":"nugen-flash-instruct","choices":[{"text":" flows","index":0,"finish_reason":null}]}

data: {"id":"nugen-abc123","object":"text_completion","created":1704123600.0,"model":"nugen-flash-instruct","choices":[{"text":" like water","index":0,"finish_reason":"stop"}],"usage":{"prompt_tokens":7,"completion_tokens":18,"total_tokens":25},"confidence_score":86.5221}

data: [DONE]

Example Request (Tokenized Prompt):

POST /api/v3/inference/completions
Headers: {"Authorization": "Bearer <api_key>"}

{
  "model": "nugen-flash-instruct",
  "prompt": [1014, 6766, 318],
  "max_tokens": 50,
  "temperature": 1
}

Notes:

• Streaming provides real-time response generation for better user experience
• The prompt parameter accepts pre-tokenized inputs for advanced use cases
• Default max_tokens is 16 - adjust based on your needs
• Temperature controls randomness: 0 = deterministic, 2 = very random
• confidence_score is only available for Domain-Aligned AI models
• Include X-Session-ID header for multi-turn conversation tracking
• Token usage is tracked and billed per request