Failure classification

Configure failure classification to categorize failed runs and choose whether to heal, warn, or fail. Review or override the diagnosis, then use AI triage to investigate and repair the test.

AI classification

Classification explains the evidence behind each category and the configured heal, warn, or fail action. Review the reasoning in the run viewer and override the category when the evidence supports a different diagnosis.

Momentic classification showing Healed and Test authorship labels with reasoning about a transient UI race — Classification identifies a test-authorship race and routes it to healing

Configure classification in ai.classification. Category actions and project-specific guidance live in Classification settings. When classification puts a failure in an application bug category and your workspace has Linear connected, the run viewer’s Create Linear issue opens an issue draft whose description is a prompt for your coding agent, with the run link, the classification reasoning, and the error. It also includes the test link, branch, commit, root cause, and steps before the failure when they’re available.

AI triage

The momentic-mobile CLI has no ai classify or ai triage commands. For mobile failures, use the momentic-maintain skill with the hosted Momentic MCP server for run history and quarantines and the mobile MCP server for run artifacts. Run tests into a results directory, then triage failures from that directory:

npx momentic run --output-dir test-results
npx momentic ai triage test-results --yes --timeout-minutes 10

Triage can also revise a classification while it works. If live browser evidence disproves the original diagnosis, it saves the corrected category whether or not the repair succeeds, unless you pass --no-save. Triage never overwrites a category you set by hand in the run viewer. A failure that was transient or came from outside your test can end with No test change: a successful outcome that keeps the test as it is. Triage delivers a repair only when it changes the test or a module, so it does not open a pull request when those files are unchanged. A failing run identifies the failed step and suggests the triage command:

[2/2] ✘ slow-nav.test.yaml (38s)

  1) slow-nav.test.yaml

    Assertion failure

    An AI check failed for a legitimate reason, such as a missing element, a
    change in the page structure, or an unexpected state (e.g. loading state).

    Failed at:

        5 |   - click: the Submit button
     >  6 |   - assert:
        7 |       that: The error message "Your password is invalid!" is visible

    https://app.momentic.ai/runs/cef3a8f6-8347-406d-b496-ac43c71d627f

  Tests     2 failed | 0 passed (2)
  Duration  67s

  Run momentic ai triage to attempt automatic fixes · https://momentic.ai/docs/reliability/failure-classification#ai-triage

Triage reads the failing runs, tests candidate fixes in a live browser, and reports each repair with its reasoning:

$ npx momentic ai triage test-results --yes --timeout-minutes 10
Momentic CLI v3.57.0

Healing 2 tests · 1 worker
  source  ./test-results

Triage
[tool call] momentic_get_run(runId: 06189c5d-...)
[tool call] momentic_get_step_result(runId: 06189c5d-..., includeAiInput: true)
[tool call] preview_step(step: --step-type TYPE --description "the Username input" ...)
[tool call] run_step(fromStep: ..., toStep: ...)

  ✓ wrong-assert.test.yaml (1m03s)
    Proposed solution: replace the unsupported greeting assertion with the
    minimum positive-login journey. The main flow now fills the Username input
    with student, fills the Password input with Password123, clicks Submit, and
    retains a single semantic AI assertion confirming a successful login.

  ✓ slow-nav.test.yaml (53s)
    Proposed solution: insert two focused TYPE steps before the existing Submit
    action: fill the Username input with student, then fill the Password input
    with incorrectPassword. The existing assertion was retained unchanged.

  Tests     2 healed (2)

The slow-nav repair was written to disk as:

slow-nav.test.yaml

 steps:
+  - fill:
+      text: student
+      into: the Username input
+  - fill:
+      text: incorrectPassword
+      into: the Password input
   - click: the Submit button
   - assert: The error message "Your password is invalid!" is visible

The test previously submitted an empty form, so the page showed the username-invalid error and the password-invalid assertion could never pass. Triage restored the intended login flow instead of weakening the assertion. For an interactive investigation, install the coding agent skills and the MCP server, then invoke momentic-maintain with the run ID. To keep CI passing while a heal is pending, see Let the verdict gate CI. For CI, make sure triage runs even when the test step fails. Each repair attempt starts a browser, so if browsers run locally, size the runner for up to one browser per --parallel worker (one by default):

.github/workflows/momentic.yml

- name: Run tests
  continue-on-error: true
  run: npx momentic run --output-dir test-results

- name: Triage failing tests
  if: always()
  run: npx momentic ai triage test-results --yes --timeout-minutes 10

The complete auto-heal workflow includes authentication, permissions, checkout, and browser setup. See the GitHub Actions guide or another CI integration for runner setup. The momentic ai reference covers alternative run selectors, queue filters, parallelism, delivery overrides, and exit codes.

Classify during the run

In-flow classification runs the classification agent inside momentic run, when a run exhausts its retries. The CLI includes a category, reasoning, and triage action when the run exits, so CI does not need a separate classification step. Prefer this over invoking momentic ai classify afterwards.

Enable it

Turn it on in momentic.config.yaml:

momentic.config.yaml

ai:
  classification: true

The toggle is versioned with your tests and applies to local and CI runs. See the AI configuration reference for every field.

What happens on a failure

When a run fails after all retries, the CLI shows a Classifying failure step and the agent inspects the failed step, screenshots, error details, evidence from past runs, and git context when available. The CLI writes the verdict into the run’s results:

The category, reasoning, confidence, and recoverability appear in the CLI output and in the run viewer’s failure details.
The category’s configured action (heal, warn, or fail) is attached to the run, so downstream steps such as momentic ai triage and momentic results check can act on it without re-classifying.
With transient failure recovery enabled, a failure the classifier judges one-time recoverable triggers an in-run recovery attempt guided by its reasoning.

Failure categories, their actions, and the optional custom prompt stay cloud-managed in Settings > Classification.

Let the verdict gate CI

By default the classifier only records a category. Every failed test still fails CI. Set overrideExitCode: true so failures routed to heal or warn exit 0. The job passes while the failure is repaired or reported. Failures routed to fail (for example BUG) still exit 1.

momentic.config.yaml

ai:
  classification:
    enabled: true
    overrideExitCode: true

To opt in for a single invocation instead, pass --classify-override-exit-code to momentic run. To skip classification for one invocation, pass --no-classify.

In-flow vs `momentic ai classify`

momentic ai classify runs the same agent as a separate command against already-finished runs. Prefer the config-driven in-flow setup for CI:

The in-flow setup needs no separate CI command or run selector. The verdict is part of the run result when momentic run exits.
The exit-code override can only be applied while the run is still deciding its exit code, so overrideExitCode works only in-flow.
The CLI saves the verdict automatically; ai classify persists only with --save.

Reach for ai classify when you want to re-classify a past run, classify runs from an older CLI version, batch-classify a run group or commit, or ask follow-up questions with --interactive.

Get started

Mo

Regression tests

App setup

Write tests

Run tests

Maintain tests

Integrations

Workspace

Migrate and upgrade

Failure classification

AI classification

AI triage

Classify during the run

Enable it

What happens on a failure

Let the verdict gate CI

In-flow vs `momentic ai classify`

AI classification

AI triage

Classify during the run

Enable it

What happens on a failure

Let the verdict gate CI

In-flow vs momentic ai classify

In-flow vs `momentic ai classify`