> ## Documentation Index
> Fetch the complete documentation index at: https://momentic.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Performance

> Runtime characteristics, step-by-step timings, how to speed up large test suites with parallelism and sharding, and Momentic vs. Playwright benchmarks.

## Speeding up large test suites

By default the CLI runs tests one at a time on a single machine, so a big test
suite runs as slowly as the sum of its tests. Two independent knobs cut
wall-clock time:

* **Parallelism (one machine):** `--parallel <n>` runs `n` tests at once, each
  in its own browser instance. Raise it until the runner's CPU/memory is
  saturated.

  ```bash theme={null}
  npx momentic run --parallel 4
  ```

* **Sharding (many machines):** split the test suite across runners with
  `--shard-count` and a per-runner `--shard-index`, then combine the outputs
  with `momentic results merge` before uploading. This is how you turn a
  multi-hour sequential run into minutes across a CI matrix.

  ```bash theme={null}
  # runner 1 of 4
  npx momentic run --shard-index 1 --shard-count 4 --output-dir test-results/shard-1
  ```

Combine both: shard across runners, and use `--parallel` within each runner.

<Tip>
  Sharding and parallelism multiply: 4 shards x `--parallel 4` is up to 16 tests
  running at once. See the [GitHub Actions
  guide](/running-tests/ci/github-actions#sharding) for a ready-to-copy matrix +
  merge workflow, and [`run`](/cli-reference/momentic/commands/run#sharding) /
  [`results merge`](/cli-reference/momentic/commands/results) for every flag.
</Tip>

Also make sure [step caching](/reliability/step-cache) is saving in CI
([`--save-cache`](/running-tests/ci/custom-setups#saving-caches)): cached steps
run in well under a second, so a warm test suite is much faster than a cold one.

### Mobile test suites

Mobile runs use the same `--parallel` / `--shard-count` / `--shard-index` flags
through the `momentic-mobile` CLI, with a few platform caveats:

* `--parallel AUTO` saturates the current shard. On **remote** emulators each
  test gets its own session, so parallelism is safe and any org quota is
  enforced server-side.
* **Local Android** runs need a distinct AVD per parallel test: running multiple
  tests against the same `--local-avd-id` conflicts.
* **Local iOS** runs prefer `--parallel 1` because concurrent Appium instances
  share a driver manifest.

```bash theme={null}
# remote emulators, sharded across runners and parallel within each
npx momentic-mobile run --shard-index 1 --shard-count 4 --parallel AUTO
```

Merge shards with
[`momentic-mobile results merge`](/cli-reference/momentic-mobile/commands/results)
before uploading, exactly like web. See
[`momentic-mobile run`](/cli-reference/momentic-mobile/commands/run#sharding)
for every flag.

## Execution speed

Runtime depends on how a test is written and the state of the system under test.
A Momentic test that only aims for feature parity with Playwright or Cypress
runs at about the same speed. Thanks to [step caching](/reliability/step-cache),
over 99% of steps execute in under 500ms:

| Preset action                      | Average runtime |
| ---------------------------------- | --------------- |
| **Click**                          | 250ms           |
| **Type**                           | 340ms           |
| Choosing from a `<select>` element | 275ms           |
| Pressing a key                     | \<5ms           |
| Scroll                             | \<5ms           |
| **Page check** attempt             | 220ms           |
| **Element check** attempt          | 210ms           |
| **Visual diff**                    | 620ms           |

AI-enhanced steps are slower on first run but most are cached for subsequent
runs. Approximate **first-time** runtimes:

| AI-enhanced action                              | First-time runtime range |
| ----------------------------------------------- | ------------------------ |
| Locating an element                             | 4-8 seconds              |
| Evaluating an assertion once                    | 5-8 seconds              |
| Extracting data from the page                   | 5-8 seconds              |
| Generating a single command in an **AI action** | 6-12 seconds             |
| Classifying a test failure                      | 20-30 seconds            |
| Auto-healing a section                          | 30+ seconds              |

## Benchmarks

### Overview

We have published a basic benchmark comparing Momentic against Playwright in
this publicly accessible test automation
[environment](https://practicetestautomation.com/practice-test-login/).

The results illustrate that cached Momentic steps are **only 52ms slower** on
average than comparable Playwright functions. Non-cached steps that require AI
to execute run on average **6354ms** slower. Over 99% of all steps executed on
the Momentic platform are cached.

Note that this benchmark does not exhaustively test all Momentic step types,
many of which do not have analogs in Playwright, Cypress, or any traditional
tooling (e.g. **AI check**, **Visual diff**).

### Method

We built a Momentic test that logs into the practice automation site, as well as
an equivalent Playwright script that performs the same sequence of actions. We
obtained three different sets of measurements:

* The "Steps only" category only measures the time spent executing steps in both
  software.
* The "End-to-end" category includes Momentic's fixed bootstrap (e.g. API key
  check) and test result upload times. For Playwright, the end-to-end time
  includes CLI initialization time but does not involve any upload of data.
* The "First-run" category ran with caching explicitly disabled and thus
  includes the runtime of 4 fresh AI completions.

All measurements were completed on a M3 Max Macbook Pro with 36GB RAM running
macOS Sonoma.

### Results

All values are P50 milliseconds measured over 10 independent samples.

|                      | Playwright | Momentic |
| -------------------- | ---------- | -------- |
| Steps only           | 961ms      | 1173ms   |
| End-to-end           | 1870ms     | 3998ms   |
| First-run steps only | N/A        | 26379ms  |

The source used for this benchmark is provided below:

<CodeGroup>
  ```yaml benchmark.test.yaml theme={null}
  fileType: momentic/test/v2
  id: log-in-practice-test-automation
  url: https://practicetestautomation.com/practice-test-login/
  retries: 1
  steps:
    - type:
        text: student
        into: the username input
    - type:
        text: Password123
        into: the password input
    - click: the submit button
    - checkPageContains: Logged In
  ```

  ```javascript benchmark.spec.js theme={null}
  import { chromium } from "playwright";

  async function main() {
    console.log("starting steps", Date.now());
    const browser = await chromium.launch({
      headless: true,
    });
    const context = await browser.newContext();
    const page = await context.newPage();
    await page.goto("https://practicetestautomation.com/practice-test-login/");
    await page.locator("#username").fill("student");
    await page.locator("#password").fill("Password123");
    await page.locator("#submit").click();
    await page.getByText("Logged In Successfully").waitFor({ state: "visible" });
    console.log("done", Date.now());
    process.exit(0);
  }

  void main();
  ```
</CodeGroup>