Skip to main content
POST
Create Evaluation

Authorizations

Authorization
string
header
required

Your Nugen API key, sent as a Bearer token

Body

application/json

Start an evaluation of a model, or a comparison of two models, on a benchmark.

model_id
string
required

ID of the model to evaluate

Example:

"model_01k4x9m2p7q3r9d1"

benchmark_id
string
required

ID of an existing benchmark

Example:

"benchmark_01k4x9m2p7q3r6a1"

evaluation_name
string | null

Display name for the evaluation. Defaults to the evaluation id if omitted.

Example:

"Clause Review v2 Eval"

custom_metrics
EvaluationMetricsConfig · object | null

Built-in metrics to run and up to 5 custom metrics. Both built-in metrics run when omitted.

baseline_model_id
string | null

ID of baseline model for comparison mode (eval-compare)

Example:

"model_01k4x9m2p7q3r9d2"

Response

Returns a unique identifier for the initiated evaluation along with the initial evaluation status. This endpoint starts an asynchronous evaluation using a specified benchmark and model(s), allowing users to track progress and retrieve results once completed.

Acknowledgement of a new evaluation.

evaluation_id
string
required

Unique identifier for the evaluation

Example:

"evaluation_01k4x9m2p7q3r7b1"

evaluation_name
string
required

Display name of the evaluation

Example:

"Clause Review v2 Eval"

model_id
string
required

ID of the model being evaluated

Example:

"model_01k4x9m2p7q3r9d1"

status
enum<string>
required

Current status of the evaluation

Available options:
CREATED,
QUEUED,
PROCESSING,
READY,
FAILED,
STOPPED
Example:

"PROCESSING"

benchmark_id
string
required

Benchmark ID used

Example:

"benchmark_01k4x9m2p7q3r6a1"

created_at
string<date-time>
required

timestamp of evaluation creation

Example:

"2024-02-24T10:00:00Z"

message
string
required

Status message

Example:

"Evaluation created successfully"