Test checks

Ask one of the local check's five model questions

POST /api/v1/tests/checks/{check_id}/questions

Asks one of a local check's model questions. Requires sign-in, model_api and an Idempotency-Key. Questions of a started check do not count again and continue through cap, month and plan changes, within the replay window and question bound. Older uncounted checks count when their first question receives a place. Queue mode runs up to four questions per person at once across their keys, agents and checks.

Six input kinds have exact v17 prompts. The service scrubs again, rejects oversized input without shortening it, and keeps IDs, hashes and scores for replay. The configured prompt token limit also determines the byte limits for the rendered prompt, the total input fields and the question body; stored answers can replay after those limits are lowered. Only Free accounts that opt in to sharing data to help improve Commitscale keep scrubbed input fields in a separate store; withdrawal deletes them. Responses echo IDs.

Retries must preserve the exact body, request_id and key. An expired replay never starts another charge. A model_unavailable problem leaves the question unanswered. Answer Yes means p_yes >= 0.5; it does not apply the CLI's policy thresholds.

Parameters

check_idpath · required · string

The check's random UUID.

Idempotency-Keyheader · required · string

Required for model checks and questions. Reuse the exact body and IDs on retries. Keys are scoped to the caller's namespace and retained as hashes after the 24-hour replay window. Expired identities return model_replay_expired and cannot count again.

Minimum length: 1 · Maximum length: 255

Example request

curl --request POST 'https://commitscale.com/api/v1/tests/checks/019a1f40-7c00-7000-8000-000000000001/questions' \
  --header "Authorization: Bearer $CSCALE_CHECK_KEY" \
  --header 'Idempotency-Key: 019a1f40-7c00-7000-8000-000000000003' \
  --header 'Content-Type: application/json' \
  --data '{
  "candidate_id": "candidate:1",
  "input": {
    "hunk": "@@ -1 +1 @@\n-return total;\n+return total + tax;\n",
    "kind": "hunk",
    "path": "src/total.rs"
  },
  "prompt_version": "v17-layout-1",
  "question": "behaviour",
  "redaction_version": "1",
  "request_id": "019a1f40-7c00-7000-8000-000000000002"
}'

Example values illustrate the format. Use your own credentials and identifiers.

Request body

application/json

candidate_idrequired · string

Opaque candidate identifier, never a path or excerpt.

inputrequired · object

Input for this question. Its kind must match the question; excerpts must already have sensitive values removed.

Fields of input

Scrubbed code or test evidence for the selected model question.

One of these shapes

hunkrequired · string: Changed lines as a unified diff hunk, with sensitive values removed.
kindrequired · hunk: Selects the hunk shape.
pathrequired · string: Repository-relative file path. Do not send absolute machine paths.

coderequired · string: Relevant source code around the proposed mistake, with sensitive values removed.
editrequired · string: The proposed edit, with sensitive values removed.
kindrequired · mistake: Selects the mistake shape.
pathrequired · string: Repository-relative file path. Do not send absolute machine paths.

hunkrequired · string: One @@ hunk header and its body, without diff file headers; sensitive values removed.
kindrequired · test_code: Selects the test_code shape.
pathrequired · string: Send (path hidden). This question must not use the file name as evidence.

code_diffrequired · string: Source change as a unified diff, with sensitive values removed.
editrequired · string: The proposed edit, with sensitive values removed.
kindrequired · test_edit: Selects the test_edit shape.

kindrequired · build: Selects the build shape.
logrequired · string: The end of the failed test run's log, with sensitive values removed.

code_diffrequired · string: Source change as a unified diff, with sensitive values removed.
editrequired · string: The proposed edit, with sensitive values removed.
kindrequired · caught: Selects the caught shape.
test_diffrequired · string: Test change as a unified diff, or an empty string if no tests changed.

prompt_versionrequired · string

Prompt layout version. Send v17-layout-1.

questionrequired · behaviour / test_code / test_edit / build / caught

One of the five questions: behaviour, test_code, test_edit, build or caught.

redaction_versionrequired · string

Version of the client's secret-scrubbing rules. The server scrubs the input again.

request_idrequired · string (uuid)

Random UUID for this question, reused for exact retries.

Responses

200 First-token Yes/No probability, or the stored answer.

application/json

{
  "answer": "Yes",
  "candidate_id": "candidate:1",
  "check_id": "019a1f40-7c00-7000-8000-000000000001",
  "checkpoint": "v17-f16:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
  "model_version": "v17-f16",
  "p_yes": 0.98,
  "prompt_sha256": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
  "prompt_version": "v17-layout-1",
  "question": "behaviour",
  "request_id": "019a1f40-7c00-7000-8000-000000000002",
  "template_version": "cscale-1-chat-v1",
  "tokenizer_version": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
}

answerrequired · Yes / No

Yes at p_yes >= 0.5. The CLI applies each question's policy separately.

candidate_idrequired · string

Opaque candidate identifier. Do not put a path or source excerpt here.

check_idrequired · string (uuid)

Random UUID for this check, reused for exact retries.

checkpointrequired · string

Identifies the model that answered: its version, a colon, and a SHA-256 checksum.

Pattern: ^[^:]+:[0-9a-f]{64}$

model_versionrequired · string

The served model's name, such as v17-f16.

p_yesrequired · number (double)

Yes and No normalized against each other; not a policy threshold.

Minimum: 0 · Maximum: 1

prompt_sha256required · string

SHA-256 of the scrubbed, rendered chat input, as UTF-8 bytes.

Pattern: ^[0-9a-f]{64}$

prompt_versionrequired · string

Prompt layout version. Send v17-layout-1.

questionrequired · behaviour / test_code / test_edit / build / caught

One of the five questions: behaviour, test_code, test_edit, build or caught.

request_idrequired · string (uuid)

Random UUID for this question, reused for exact retries.

template_versionrequired · string

Chat template used to render this question, currently cscale-1-chat-v1.

tokenizer_versionrequired · string

SHA-256 of the pinned tokenizer.json.

Pattern: ^[0-9a-f]{64}$

RateLimit and RateLimit-Policy describe the request rate. Follow Retry-After when a response asks you to wait.

Problem responses

Errors use application/problem+json. Act on the stable code, and follow next when present. See the problem format.

model_check_cap_reached403

The Free allowance of 100 checks this UTC calendar month is used across Free workspaces. Verified addresses that reach one inbox share the allowance. The check was not started.

Do not retry automatically. Open next.url to upgrade the workspace to Pro and keep checking. Already started checks may finish.

model_unavailable503

The local check's model is not configured or could not return both answer probabilities.

Leave the question unanswered. Retry the exact body with the same check ID, request ID and idempotency key within the remaining check budget.

excerpt_too_large413

A model request exceeds a field, line, rendered byte, token, body or internal per-check question bound.

Skip the question and leave it unanswered. Do not shorten a diff, split its meaning across questions or retry the same oversized input.

model_replay_expired410

A model check's 24-hour replay window expired. Its retained identities cannot be charged again.

Leave this question unanswered. Never silently substitute a fresh check for an expired retry.

check_key_scope403

A check-only key may call the test check, read its own usage and acceptance rule, contact support, and withdraw data sharing.

Use a person's sign-in for account management. More agent power needs the owner's passkey; this key cannot widen itself.

feature_off403

The namespace plan does not include this feature.

Retrying on the same plan will not help. GET /api/v1/namespaces/{namespace}/plan, and the namespace's Plan page in Settings, show what its plan includes.

unauthenticated401

The request carried no credentials.

Use this operation's sign-in method. Bearer credentials use Authorization: Bearer <token>; browser credentials use the session cookie.

invalid_token401

The token is unknown, expired, or revoked, its agent was revoked, or an OAuth token was used outside its MCP resource.

Stop using it. Sign in again, or ask the owner to create a replacement agent key.

forbidden403

The caller may not do this: only people may, only owners may, or the agent was not given this repository or write access.

Stop. Retrying will not help. Ask the owner for the access, or leave the step to a person.

not_found404

It does not exist, or the caller cannot read it: a repository the caller cannot read is reported as not found.

Check the name or ID, and that the token has the repository.

invalid_request400 / 413 / 422

The request is malformed or breaks a rule, such as a title that is too long, or a body that is not the expected JSON.

Fix the request as detail says. The same request fails the same way.

quota_exceeded429

With 429, the caller used its requests for the minute. With 422, a count limit was reached.

With 429, wait Retry-After seconds and retry. With 422, remove one first.

idempotency_key_reused409 / 422

The idempotency key was already used for a different request.

Use a new key for each new request, and the same key only for a retry of the same request.

idempotency_key_in_use409

The first request with this key is still running, including a queued model question, or it succeeded with a secret, such as a new token, which is never replayed.

When Retry-After is set, retry then with the same key. Otherwise the first request succeeded: do not repeat it.

internal500

The server failed, and logged why.

Retry after Retry-After seconds. A write is safe to retry with the same idempotency key.

unavailable503

With 502, a service the server needs did not answer, such as a sign-in provider. With 503, the server is busy, such as when a workspace is locked or host lookups are full, or a write with an idempotency key runs long: that write goes on, and its answer is kept for the key.

With 502, retry later, or sign in another way. With 503, retry after Retry-After seconds, using the same idempotency key for a write; a long-running write can then replay its answer.