Browse documentation

Guide

Use the test check

Check committed work, read the findings, and rerun after a fix.

Check a committed change

Run this inside your repository after committing your work and setting its target as in getting started:

cscale tests check

Checks can take minutes. With command timeouts, run cscale tests check in the background and wait for its exit status. Running it again resumes saved work when the inputs still match.

The check uses private copies and the machine's installed build tools. It executes tests and experiments to find behavior that the tests do not protect. CScale 1 answers focused questions about relevant code excerpts. The trusted target branch supplies the check settings. There is no fixed time limit: the check scales its limits with the project and the machine.

Set a test command

The check discovers test commands from your project. From CLI 0.2.9, it follows common JavaScript wrapper scripts to Vitest, Jest, Mocha or node --test by itself. For example, a project's package.json might have:

{
  "scripts": {
    "test": "pnpm run \"/^test:.*/\"",
    "test:unit": "vitest run"
  }
}

When discovery cannot follow a wrapper, or your project needs a different command, put an override in .commitscale/tests.yaml on the trusted target branch. For the example above, this runs the unit script with the Vitest adapter:

version: 1
packages:
  - root: .
    adapter: vitest
    command: [pnpm, run, "test:unit"]

The root is relative to the repository, and the command is an argument array, not shell text. The adapter adds the arguments it needs to collect results; the command must accept them. Commit the file to the target branch before running the check. Use a different adapter or a configured report if your runner needs one.

What CScale 1 suspects

The check asks CScale 1 whether tests catch deliberate mistakes. It makes small mistakes in each new function and changed hunk, such as a flipped comparison or a dropped return. It proves up to five of the likeliest misses by running your tests against them, and each new file gets one of those proofs even if the check runs past its usual time:

  • A proven miss is a finding: your tests passed with the mistake in place.
  • A mistake CScale 1 is at least 80% sure your tests would miss, but that wasn't proven, is reported as a suspicion. Where its accuracy has been measured, the suspicion says how often such suspicions are wrong.
  • A hunk in a file no test reaches, such as configuration or a script, is reported when CScale 1 is at least 90% sure it changes behavior.

Suspicions never block a push. Read each one, then add a test or say why none is needed.

If a test fails once and passes when retried, the check sets it aside, names it as unstable in the report, and carries on with the rest.

The check command describes all options. To inspect the scrubbed model requests before they are sent:

cscale tests check --show-requests

Repair and rerun

Read each finding and the overall result. Fix failing tests or add a test for the uncovered behavior. Commit the fix and run the full check again. A build failure, missing tool, unanswered model question or incomplete experiment is not evidence that the change is protected.

An incomplete check means some of the work needed for the result did not finish. It can still show findings, but it is never a pass. Read the reported problem and repair the missing work before relying on the check.

If a control needs its own locked install, the base and head may have different dependency inputs. Make separate clean worktrees at the base and head commits. In each, install that revision's locked dependencies and run the project's full test command. For example, a project may use uv sync --locked, npm ci, or pnpm install --frozen-lockfile; include any test groups or other setup the project needs. These runs give you separate evidence, but do not turn the incomplete check into a pass.

From CLI 0.2.10, a lockfile change to only the project's own version doesn't need a separate install. A real dependency change still does, because one machine-mode check can't supply both installed environments. Where the runner supports it, the message gives the commands to prepare one. If the project can run its test command in disposable copies with the needed tools and dependencies available offline, try cscale tests check --isolated. It does not copy the installed environment and may also remain incomplete. Keep and report the incomplete result if the control still cannot run.

For an agent or script, request a structured report:

cscale tests check --json

Read the report's gate and process exit code:

  • 0: the check passed, or had nothing to run.
  • 1: the target's policy makes a finding blocking, or the check could not run because there is no account. The pre-push hook stops for blocking findings; it warns and lets the push through when there is no account.
  • 2: the check is incomplete. Some work didn't finish, so it isn't a pass. The hook shows and saves the result, then lets the push through.
  • 3: a person must approve first. The hook stops the push until they do.

Do not turn an error into a successful check.

To recheck one finding, use its current ID from the report:

cscale tests check --finding 0123456789abcdef

The example ID is a placeholder. A focused check never authorizes a push. It uses the machine, as the full check does. Run a full check after the fix.

Files nothing can test

Some changed files have nothing in the repository that could test them. Examples include .gitignore, agent settings, licence files, the trust file cscale init writes, or a version bump in a build settings file. The report lists each one as not checked, with the reason, and they don't stop the push. A build settings change still runs its package's tests. Real failures still block: a timeout, a broken build, a test tool error or a failing test.

Approve sensitive files

CODEOWNERS and CI workflow files decide who approves changes and what CI runs, so they wait for a person. The target's policy can add more paths. When a change touches one, the check exits 3 and names the files.

An agent stops there, tells the person which files changed and how, and asks for explicit approval. With it, the agent runs the check or the push again with the person's exact words, and the value the stop message gives for that commit:

CSCALE_USER_APPROVAL="Yes, add the owners file" CSCALE_USER_APPROVAL_FOR="<value from the stop message>" cscale tests check

Copy the value from the message; never compute it. The report keeps the words on each file. A new commit needs a new approval. An agent never sets either variable without asking, and CI never reads them.

An addition, edit or deletion of the check's policy file itself, .commitscale/policy.yaml, also exits 3, and the person approves it the same way. The check still judges the change by the target's policy, never the edited one. Renaming the policy file, or a policy file the check can't read safely, still needs a person's review.

Accept with a reason

If a finding is acceptable, use its current ID and explain why:

cscale tests accept 0123456789abcdef --reason "Covered by the integration suite."

Acceptance reads the newest saved result for the current commit and records the agent and its reason; it runs no tests. If there is no saved result yet, run the check first. The next check applies the acceptance. The target's policy determines which acceptances count. If passkey confirmation is enabled, a person must confirm. An acceptance never clears a policy flag. Do not change the policy to clear a finding.

The pre-push hook checks the commits being pushed before they leave your machine, with the normal machine check, and reuses a complete saved result for the same commit and inputs. Getting started installs it.