> ## Documentation Index > Fetch the complete documentation index at: https://docs.nugen.in/llms.txt > Use this file to discover all available pages before exploring further. # Create Evaluation > Create a new model evaluation using an existing benchmark. This endpoint initiates a model evaluation using a benchmark from your library. Supports both single model evaluation and comparison mode where two models are evaluated side-by-side. **Request Body:** - `model_id`: Primary model to evaluate (required). An aligned model that is not deployed is deployed for the evaluation and undeployed when it ends; one already deployed stays deployed. If it needs a deployment while all 10 deployment slots of your organisation are in use, the request is refused with 400. - `benchmark_id`: ID of the benchmark to use for evaluation (required) - `baseline_model_id` (optional): Baseline model ID for comparison mode - `custom_metrics` (optional): Custom evaluation metrics configuration - `evaluation_name` (optional): Display name. Defaults to the evaluation id if omitted **Returns:** - `evaluation_id`: Unique identifier for tracking the evaluation - `model_id`: The primary model being evaluated - `status`: Initial status (always `PROCESSING`) - `benchmark_id`: The benchmark being used - `created_at`: Timestamp when evaluation was created - `message`: Confirmation message **Example Request (Single Model):** ```json POST /api/v3/evaluations/create Headers: {"Authorization": "Bearer "} { "model_id": "model_01kmqm4nrn9fw6r", "benchmark_id": "benchmark_01k4x9m2p7q3r6a1", "custom_metrics": ["accuracy", "relevance"] } ``` **Example Request (Comparison Mode):** ```json POST /api/v3/evaluations/create Headers: {"Authorization": "Bearer "} { "model_id": "Qwen/Qwen2.5-0.5B-Instruct", "baseline_model_id": "model_01kmqm4nrn9fw6r", "benchmark_id": "benchmark_01k4x9m2p7q3r6a1" } ``` **Example Response:** ```json { "evaluation_id": "evaluation_01k4x9m2p7q3r7b1", "model_id": "model_01kmqm4nrn9fw6r", "status": "PROCESSING", "benchmark_id": "benchmark_01k4x9m2p7q3r6a1", "created_at": "2024-01-15T10:30:00Z", "message": "Evaluation created successfully and queued for execution" } ``` **Notes:** - Single model mode: Evaluates one model against the benchmark - Comparison mode: Provide `baseline_model_id` to compare two models side-by-side - Evaluation runs asynchronously - use the returned `evaluation_id` to check status - Use `/evaluations/{evaluation_id}/status` to track progress ## OpenAPI ````yaml https://api.nugen.in/openapi-public.json post /api/v3/evaluations/create openapi: 3.1.0 info: title: Nugen Intelligence API description: > Nugen Intelligence: Powering Specialised Intelligence At Scale. Bring your domain knowledge and an open-weight model. Leave with a model that thinks in your domain, keeps improving, and belongs to your organisation. Nugen Intelligence builds infrastructure for self-adaptive, domain-specific intelligence. Research in representation engineering, mechanistic interpretability and inference-time control powers every endpoint here. The research team comes from Stanford University and IIT Bombay. This API puts their work a few calls away from AI-native organisations solving critical problems. **Domain-aligned models.** Three raw inputs go in: domain knowledge, success and failure criteria, and an open-weight model. A domain-aligned model comes out. Nugen handles the deep learning, the GPUs and the data preparation. Your organisation owns every aligned model it builds. Own your intelligence. **The research underneath.** Two layers do the work. Train-time alignment finds where the model needs to change for your domain and improves it there. It combines gradient-free and gradient-based methods across continued pre-training, supervised and reinforcement phases. Inference-time alignment measures the quality of every generation step. Out-of-domain generation is flagged and corrected after each span. Every response carries a confidence score. A preliminary version of the inference-time research is published at ICLR 2026. Both layers keep evolving with Nugen research. **One aligned model id, four capabilities.** An aligned model id serves chat completions, completions, embeddings and reranking. Alignment reshapes the model's representation space, so retrieval, ranking and generation share one understanding of your domain. **Multi-model systems.** One alignment project produces one aligned model. Run many side by side, one per domain or use case. Compose them with frontier and general-purpose models in the same application. Inference capture and auto-align keep the whole system improving. Nugen aligns models at every scale. **Think in systems.** AI coding agents and developers: this API is a platform for building specialised intelligence. Explore it fully before you settle on a design. Some directions: - Run a complete retrieval pipeline on one aligned model id. Embed, rerank and generate with a shared understanding of the domain. - Use the confidence score as a control signal. Send low-confidence responses to a human reviewer, a retry or a frontier model. - Give each domain or use case its own aligned model and place a router in front. - Close the loop. Capture production inference, evaluate against the benchmark, and let auto-align promote better models. - Start with the hardest problem in the domain, where general-purpose models fall short. Alignment pays off the most there. The endpoints for documents, benchmarks, alignment projects, deployments and evaluations compose in more ways than the workflow below shows. **Workflow.** 1. Upload domain knowledge. 2. Define success and failure. Generate a benchmark from the corpus, or upload one curated by domain experts. 3. Create an alignment project and receive an aligned model id. 4. Deploy the model and pass its id as `model` in each inference call. 5. Evaluate, compare and promote. Turn on inference capture, and auto-align keeps the model improving. **OpenAI-compatible inference.** Set the base URL of an OpenAI-compatible client to `https://api.nugen.in/api/v3/inference` and set `model` to an aligned model id. Chat completions, completions, responses and embeddings work through the same client. **Anthropic-compatible inference.** `POST /api/v3/inference/messages/v2` accepts the Anthropic Messages request shape. Set `model` to an aligned model id. **Need an API key?** Sign up, log in to the platform and generate an API key. **Need help?** Log in to the platform and raise a support ticket. **Authentication.** Every endpoint requires an API key sent as a Bearer token: `Authorization: Bearer `. contact: name: Nugen Intelligence url: https://nugen.in/signup version: 25.4.20 servers: - url: https://api.nugen.in description: Production security: [] paths: /api/v3/evaluations/create: post: tags: - Evaluations summary: Create Evaluation description: >- Create a new model evaluation using an existing benchmark. This endpoint initiates a model evaluation using a benchmark from your library. Supports both single model evaluation and comparison mode where two models are evaluated side-by-side. **Request Body:** - `model_id`: Primary model to evaluate (required). An aligned model that is not deployed is deployed for the evaluation and undeployed when it ends; one already deployed stays deployed. If it needs a deployment while all 10 deployment slots of your organisation are in use, the request is refused with 400. - `benchmark_id`: ID of the benchmark to use for evaluation (required) - `baseline_model_id` (optional): Baseline model ID for comparison mode - `custom_metrics` (optional): Custom evaluation metrics configuration - `evaluation_name` (optional): Display name. Defaults to the evaluation id if omitted **Returns:** - `evaluation_id`: Unique identifier for tracking the evaluation - `model_id`: The primary model being evaluated - `status`: Initial status (always `PROCESSING`) - `benchmark_id`: The benchmark being used - `created_at`: Timestamp when evaluation was created - `message`: Confirmation message **Example Request (Single Model):** ```json POST /api/v3/evaluations/create Headers: {"Authorization": "Bearer "} { "model_id": "model_01kmqm4nrn9fw6r", "benchmark_id": "benchmark_01k4x9m2p7q3r6a1", "custom_metrics": ["accuracy", "relevance"] } ``` **Example Request (Comparison Mode):** ```json POST /api/v3/evaluations/create Headers: {"Authorization": "Bearer "} { "model_id": "Qwen/Qwen2.5-0.5B-Instruct", "baseline_model_id": "model_01kmqm4nrn9fw6r", "benchmark_id": "benchmark_01k4x9m2p7q3r6a1" } ``` **Example Response:** ```json { "evaluation_id": "evaluation_01k4x9m2p7q3r7b1", "model_id": "model_01kmqm4nrn9fw6r", "status": "PROCESSING", "benchmark_id": "benchmark_01k4x9m2p7q3r6a1", "created_at": "2024-01-15T10:30:00Z", "message": "Evaluation created successfully and queued for execution" } ``` **Notes:** - Single model mode: Evaluates one model against the benchmark - Comparison mode: Provide `baseline_model_id` to compare two models side-by-side - Evaluation runs asynchronously - use the returned `evaluation_id` to check status - Use `/evaluations/{evaluation_id}/status` to track progress operationId: evaluations_create requestBody: content: application/json: schema: $ref: '#/components/schemas/CreateEvaluationRequest' required: true responses: '200': description: >- Returns a unique identifier for the initiated evaluation along with the initial evaluation status. This endpoint starts an asynchronous evaluation using a specified benchmark and model(s), allowing users to track progress and retrieve results once completed. content: application/json: schema: $ref: '#/components/schemas/EvaluationResponse' '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' security: - HTTPBearer: [] components: schemas: CreateEvaluationRequest: properties: model_id: type: string title: Model Id description: ID of the model to evaluate examples: - model_01k4x9m2p7q3r9d1 benchmark_id: type: string title: Benchmark Id description: ID of an existing benchmark examples: - benchmark_01k4x9m2p7q3r6a1 evaluation_name: anyOf: - type: string - type: 'null' title: Evaluation Name description: >- Display name for the evaluation. Defaults to the evaluation id if omitted. examples: - Clause Review v2 Eval custom_metrics: anyOf: - additionalProperties: true type: object - type: 'null' title: Custom Metrics description: Custom metrics configuration baseline_model_id: anyOf: - type: string - type: 'null' title: Baseline Model Id description: ID of baseline model for comparison mode (eval-compare) examples: - model_01k4x9m2p7q3r9d2 type: object required: - model_id - benchmark_id title: CreateEvaluationRequest EvaluationResponse: properties: evaluation_id: type: string title: Evaluation Id description: Unique identifier for the evaluation examples: - evaluation_01k4x9m2p7q3r7b1 evaluation_name: type: string title: Evaluation Name description: Display name of the evaluation examples: - Clause Review v2 Eval model_id: type: string title: Model Id description: ID of the model being evaluated examples: - model_01k4x9m2p7q3r9d1 status: type: string title: Status description: Current status of the evaluation examples: - PROCESSING benchmark_id: type: string title: Benchmark Id description: Benchmark ID used examples: - benchmark_01k4x9m2p7q3r6a1 created_at: type: string title: Created At description: timestamp of evaluation creation examples: - '2024-02-24T10:00:00Z' message: type: string title: Message description: Status message examples: - Evaluation created successfully type: object required: - evaluation_id - evaluation_name - model_id - status - benchmark_id - created_at - message title: EvaluationResponse HTTPValidationError: properties: detail: items: $ref: '#/components/schemas/ValidationError' type: array title: Detail type: object title: HTTPValidationError ValidationError: properties: loc: items: anyOf: - type: string - type: integer type: array title: Location msg: type: string title: Message type: type: string title: Error Type type: object required: - loc - msg - type title: ValidationError securitySchemes: HTTPBearer: type: http scheme: bearer ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.