Testing

Test suites

Group cases for batch execution and regression.

Overview

Test suites are ordered collections of test cases used for batch execution, manual runs, Operate schedules, CI/CD triggers, and API-started validation. Suites translate sprawling libraries into operational units like smoke, regression, and compliance packs.

Suite design directly affects pipeline duration and signal quality. Smoke suites should be minimal and fast; regression suites may run nightly; compliance suites align to explicit requirement ID sets referenced in audit packets.

When cases move between suites, update schedules and pipeline configs referencing suite IDs to prevent stale execution scope.

Who should read this

  • QA engineers, SREs, platform teams, and developers operating Zof Console and APIs.

Prerequisites

  • Reviewed test cases available in the library
  • Agreement on suite naming standards within the team

When to use this workflow

  • Defining deployment gate smoke scope
  • Scheduling nightly regression across staging
  • Packaging compliance validation for quarterly audits

Step-by-step procedure

Plan suite taxonomy

Define suite categories: smoke, regression, accessibility, security-regression, etc.

Document expected wall-clock duration targets per category in team runbooks.

Map each suite to audience, developers vs QA vs compliance reviewers.

Create a suite in the test library

Open Quality → Test library → Suites and select Create suite.

Enter name following convention SUITE-<APP>-smoke or SUITE-payments-regression.

Add description explaining purpose and owner team for future operators.

Add cases with explicit ordering

Add P0 cases first when smoke ordering implies critical path sequencing.

Exclude quarantined flaky cases until Test Health clears them for reinstatement.

Review case platform tags so endpoint-only cases are not in cloud-only suites.

Validate suite via staging run

Start a run from Operate → Runs selecting the suite and staging environment.

Compare duration and failure rate against targets; trim or split suite if too slow.

Capture run ID as baseline for future regression comparisons.

Wire suite to schedules and CI

Create Operate schedule for nightly regression referencing suite ID.

Configure CI pipeline step with suite ID or exported case ID list per integration docs.

Store suite ID in repository documentation next to service ownership metadata.

Maintain suite membership

Review suite membership each sprint, remove deprecated cases, add new requirement coverage.

Announce suite diffs in release notes when smoke scope changes affect gate behavior.

Version suite changes in change management when audits require frozen suites per release.

Key concepts

Organization scope
All Zof Console and API operations are isolated to your authenticated tenant.
Governed execution
Agent output and remediation follow policy packs with human approval when configured.

Best practices

  • Keep smoke suites under fifteen minutes where organizational policy allows
  • Never mix staging-only and production-executable cases without explicit environment controls
  • Publish suite change log in team channels when CI gates depend on them
  • Prefer suite triggers over ad hoc multi-select runs for repeatability

Common issues

Suite run much longer than expected
Cases may include heavy flows or serial dependencies. Split suite or lower case count for smoke.
CI references suite but runs zero cases
Suite ID typo, archived cases, or environment filter excluding all members. Verify membership and tags.

Was this page helpful?

Test suites | Zof AI Documentation