Skip to main content
An assert step states one fact about the page in plain language. The AI checks it against the page and a screenshot of the viewport, and the step passes only when the fact is clearly true. The rules from Finding elements apply, plus the ones below.

Best practices

  • Keep assertions short and specific. Aim for under 20 words; split compound assertions into multiple steps.
  • Provide context. Point at the relevant region of the page and define any non-standard terminology.
  • Remove ambiguity. The answer should be clearly true or false given the page state. When in doubt, use single quotes for exact text matches.
Pair every meaningful or state-changing action with an explicit assertion. An action completing does not prove that the product reached the intended state.
Avoid assertions on tooltips, toasts, loading spinners, and other UI that may disappear before a step evaluates it. Assert the durable side effect instead:
  • Instead of A Saved toast is visible, assert that the saved row has a green checkmark.
  • Instead of A loading spinner is visible, assert that the results table is populated.
If a transient state is the behavior you need to verify, use a run assertion for web tests. To check several independent expectations without stopping at the first failure, use soft assertions.

Good assertions

Bad assertions

Typos

The agent tolerates minor typos: john@@gmail.com matches john@gmail.com. For exact matching, wrap the expected value in single quotes and instruct the agent explicitly: the username is exactly 'john@@gmail.com'. do not allow any typos. When generating test data programmatically, prefer long, distinct values to avoid accidental matches.

Run assertions

Run assertions are plain-English statements you expect to have been true at some point during the run, including transient UI like toasts, spinners, and pop-ups that appear and disappear too quickly for a step-based assertion to catch. After the run finishes, an AI agent reviews its video recording to evaluate them. Any failed run assertion fails the run. The CLI records video for tests that define them. Run assertions work in web, Android, and iOS tests. In the mobile editor, open Run assertions in the toolbar to add, edit, or delete them.
Mobile run assertions require momentic-mobile 1.43.0 or later. If the mobile CLI can’t evaluate them, for example because the evaluator returned no results, it logs a warning and the run keeps its passed status.
The same best practices apply, with one difference: run assertions may describe a sequence (“X appears, then Y”) because the agent evaluates the whole timeline rather than a single moment.

ffmpeg requirement

Evaluating web run assertions with the CLI requires ffmpeg to sample video frames. If any test in a momentic run defines run assertions, the CLI checks for ffmpeg before executing and exits with an error when it is missing. Mobile run assertions send the recording to the model without sampling frames, so they need ffmpeg only on AWS Device Farm, which uses it to record the video. Install it with your package manager:
  • macOS: brew install ffmpeg
  • Debian/Ubuntu: apt-get install ffmpeg
  • Windows: winget install ffmpeg (or choco install ffmpeg)
The CLI uses the first ffmpeg on PATH. To use a specific binary instead, set the MOMENTIC_FFMPEG_PATH environment variable to its full path. Cloud runs evaluate run assertions on Momentic’s infrastructure and do not require a local ffmpeg. GitHub-hosted Actions runners (ubuntu-latest, macos-latest) already include ffmpeg; slim container images may not.