Skip to main content

Execution Plans

An execution plan organizes test cases into a runnable bundle. You use it to run several tests together — for example a full end-to-end flow that spans multiple test cases — and you can also schedule the plan or trigger it from your CI/CD pipeline.

A plan answers three questions:

  1. Which tests are part of it?
  2. In what order are they executed (purely parallel, purely sequential, or a mix)?
  3. How do parameter values flow from earlier tests into later tests?

You build a plan by organizing tests into groups on the Test-Cases tab. Each group can either contain hand-picked tests, or be driven by a tag (auto-populated from every test that currently carries that tag) — see Groups. The plan's group structure controls execution order and how parameter values are passed between tests.

You manage plans under Execution Plans in the left navigation. Pressing Ctrl+m on the list page creates a new plan. The plan list itself is organized into Folders — these are unrelated to the groups inside an individual plan described below.

Tabs

An execution plan has four tabs:

TabWhat it is
GeneralTitle, description, the schedule, and setup/teardown.
Test-CasesThe group/test structure and parameter bindings.
ExecutionsPaginated history of every run of this plan, with status, duration and triggering source. Click a row to open the execution details.
Change HistoryAudit trail of changes to the plan definition, with the same behaviour and limitations as the test Change History.

Header actions

The top right of the plan details page exposes:

  • Start — runs the plan immediately with its current configuration. If the plan has multiple groups, the button opens a small menu so you can also start from a specific group (useful for resuming a failed run without re-running the earlier groups).
  • Pipeline — opens the CI/CD configuration for triggering this plan from outside msg.ZenTestAI. See Pipeline integration below.
  • Save / Delete — bottom action bar. Delete is permanent.

General tab

  • Title — required.
  • Description — optional.
  • Setup — runs before the first group. Can be either a saved test case or a script (pick one or the other, not both). Optional.
  • Teardown — runs after the last group, with the same script-or-test choice. Optional.
  • Schedule — see Scheduling below.

Scheduling

Three modes:

  • Manual — the plan only runs when someone clicks Start or triggers it via the pipeline.
  • Time Based — By Duration — pick a recurrence (every N minutes, hours, days, weeks or months).
  • Cron — enter a cron expression. The editor validates the expression as you type (with a short debounce) and shows a preview of the next few scheduled runs in human-readable form. Cron expressions are interpreted in the server's local time; the editor displays the offset so you can see what that means in your local time zone.
warning

Frequent schedules can become expensive — every run consumes AI tokens and runner capacity. Pick the lowest cadence that meets your need, especially for cron plans.

Notifications, retries on failure and fail-fast are not configured on the plan today.

Test-Cases tab

The Test-Cases tab is where you give the plan its structure. Its layout is two-column:

  • Left — the groups that make up the plan.
  • Right — a searchable, folder-organized list of every test in the tenant. You drag tests from the right onto the left.

The execution semantics are simple:

Tests within a group run in parallel. Groups run sequentially, one after another.

Groups

Each group has:

  • A drag handle to reorder groups.
  • A name field (optional).
  • An optional Tag chip. Clearing the chip makes the group static; selecting a tag makes the group tag-driven.
  • An Execution variant indicator (visible mainly on tag-driven groups; on static groups variants are picked per test — see below).
  • A delete button.

Groups come in two flavours:

  • Static groups — contents are chosen by hand. Drag tests from the right panel onto the group. You can reorder tests inside the group, but order has no effect on execution (everything in the group runs in parallel).
  • Tag-driven groups — contents come from whichever tests carry the selected tag. The group is read-only: you cannot drag tests in or out, and the title is locked. Removing the tag converts the group back to static and clears its tests.

Static and tag-driven are mutually exclusive — a group is one or the other.

To add a new group, drop a test onto the dashed "drop here to create a group" zone between existing groups, or click Add new group at the bottom.

Picking the execution variant

If a test has variants, you can pick which variant runs inside the plan. For static groups this is set per test in the group; tag-driven groups always run each test with its default variant.

Parameter bindings

Each test inside a static group has a small parameter chip that opens the Parameter Bindings dialog. The dialog lists every parameter of that test, with one of four binding modes per parameter:

ModeWhat it does
Map by nameLooks for a parameter with the same name in an earlier group's tests and uses the value captured there. If no earlier test produces it, the run fails.
From execution variantUses the value the parameter has in the test's selected variant (i.e. the test's own definition).
HardcodedA fixed value entered directly into the binding dialog. Useful for plan-only constants without changing the underlying test variants.
From a specific source testPick a specific earlier test in the plan and a specific parameter name on that test. Use this when two earlier tests both expose a parameter with the same name and you need to disambiguate.

A few important rules:

  • Bindings can only read backwards. A test in group 3 can bind to parameters captured in groups 1 or 2 — never from a later group or from a sibling running in parallel.
  • Parallel tests in the same group do not share parameter values; each parallel test has its own scope.
  • New tests added to a group default to Map by name for every parameter. Review the bindings before saving — implicit name-matching is easy to get wrong when two unrelated tests happen to share a parameter name.

There is no plan-level constants section. Use Hardcoded bindings on individual tests to inject fixed values, or set the value on the test's variant.

Pipeline integration

The Pipeline button takes you to the product administration page with the pipeline configuration opened for this plan. From there you can copy the webhook URL and authentication token you need to call the plan from your CI/CD system (GitHub Actions, GitLab CI, Jenkins, Azure DevOps, …).

A run triggered from the pipeline appears on the Executions tab just like a manual run, but with its triggering source marked accordingly.

Executions tab

The Executions tab is a chronological table of every run of this plan, with:

  • when it started,
  • how long it took,
  • the final status,
  • the triggering source (manual / scheduled / pipeline).

Click a row to open the execution details, where you can inspect each test that ran in the plan, see screenshots and logs, and re-run individual tests.

Limitations

Things the plan does not offer today, in case you go looking for them:

  • No plan-level parameter constants — use Hardcoded bindings or set the value on a test variant.
  • No per-group execution variant override on static groups — pick the variant per test.
  • No e-mail notifications, automatic retries, or fail-fast between groups configured at the plan level.
  • No rollback from the Change History tab — see Change History.