Run AI evaluation suite
POST/ai/copilot/evals
Triggers execution of an evaluation suite, running all test cases against the configured AI model and scoring the outputs according to the suite's criteria. Results are stored asynchronously; poll the /evals/{id} endpoint to check completion. Use to validate prompt changes or measure regression after a model update.
Request
Request body
Content type application/json (required).
| Field | Type | Required | Description |
|---|---|---|---|
suite_id | integer | Yes | ID of the evaluation suite to execute. |
model_override | string, nullable | No | Optional model identifier to override the tenant default for this run. |
notes | string | No | Optional label or notes for this run (e.g. "pre-deployment baseline"). |
Example
{
"suite_id": 5,
"model_override": "claude-3-haiku-20240307",
"notes": "Baseline run before template v2 rollout."
}Responses
202 Evaluation run accepted and queued for processing
Content type application/json, object · AiEvaluationRunV1.
| Field | Type | Required | Description |
|---|---|---|---|
id | integer | No | Auto-incremented primary key. |
suite_id | integer | No | ID of the suite that was executed. |
status | string | No | Current execution status of the evaluation run. One of: |
score | number, nullable | No | Weighted aggregate score from 0 to 100. Null while still running. |
pass_count | integer, nullable | No | Number of test cases that met their scoring thresholds. |
fail_count | integer, nullable | No | Number of test cases that failed their scoring thresholds. |
started_at | string (date-time), nullable | No | |
completed_at | string (date-time), nullable | No | |
created_at | string (date-time) | No |
Example
{
"id": 31,
"suite_id": 5,
"status": "queued",
"score": null,
"started_at": null,
"completed_at": null,
"created_at": "2026-04-08T11:00:00Z"
}422 Validation failed — check suite_id
Example request
curl -X POST "https://{tenant}.faciotech.net/api/v1/ai/copilot/evals" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-d '{"suite_id": 5, "model_override": "claude-3-haiku-20240307", "notes": "Baseline run before template v2 rollout."}'