Testing
Test execution
Run validation through the Console, schedules, and Agent Console.
Overview
Test execution in Zof spans Console runs, Operate schedules, Agent Console cloud agents, endpoint agents on customer infrastructure, and programmatic triggers via API or CLI. All paths share orchestration, telemetry, and results storage, differences are routing, environment, and policy.
Choosing the correct execution path prevents false infrastructure errors: web smoke tests via cloud agents, desktop flows via endpoint agents, scheduled regression during off-peak windows, pipeline gates on merge queues.
Execution status vocabulary is consistent across surfaces: Queued, Running, Passed, Failed, Cancelled, Error. Train teams to treat Error as platform investigation and Failed as product or test logic investigation.
Who should read this
- QA engineers, SREs, platform teams, and developers operating Zof Console and APIs.
Prerequisites
- Reviewed cases or suites ready for execution
- Application registered with correct environment metadata
- Agents online with matching labels when using Agent Console
When to use this workflow
- Onboarding new team members to Zof terminology and workflows
- Authoring internal runbooks aligned with Console labels
- Designing CI/CD or webhook integrations against documented behavior
Execution routing decision
Execution routing decision
Select path based on application type, network boundaries, and operational urgency.
Step-by-step procedure
Confirm inventory and environment
Verify cases are reviewed and suites reflect intended scope.
Select staging vs production environment explicitly, never assume default is safe.
Check maintenance windows that might affect application availability.
Launch Console run
From Operate → Runs or project detail, start New run with suite or case selection.
Confirm agent routing preview shows cloud or endpoint as expected.
Monitor Queued → Running transitions; investigate prolonged Queued states in Agent Console.
Execute via Agent Console
Open Automation → Agent Console for fleet visibility before critical executions.
Verify agent heartbeat, capability advertisements, and label match for target environment.
Launch execution from Agent Console when operators need co-located telemetry with agent logs.
Configure schedules
Create Operate schedule with cron or interval appropriate to regression depth.
Attach notification channels for failure thresholds to on-call rotations.
Pause schedules during known infrastructure migrations.
Trigger from CI/CD
Use API or integration to start runs on merge or deployment events.
Capture returned run ID in pipeline logs for correlation with Console detail.
Fail pipeline on Failed status according to policy; do not ignore Error statuses.
Handle terminal states
Passed: archive evidence if release gate requires run ID attachment.
Failed: triage case results, open defects, consider Test Health clustering.
Error: engage platform operators; check agent connectivity and orchestration logs.
Cancelled: document reason; re-run only affected scope when resuming.
Key concepts
- Cloud agent
- Executes web/API validation in Zof-managed runtimes.
- Endpoint agent
- Executes on customer-controlled machines for desktop or private network apps.
- Queued
- Awaiting capacity or agent assignment.
- Error
- Orchestration or infrastructure fault, not a failed assertion.
Best practices
- Run smoke on every deployment; defer full regression to schedules
- Always log run ID in change tickets
- Separate endpoint and cloud suites to avoid misrouted jobs
- Align secret injection with organizational policy, never embed in case steps
Common issues
- Jobs stay Queued
- Offline agents, label mismatch, or concurrency limits. Inspect Agent Console fleet health.
- Passed run with ignored cases
- Suite filter or skipped cases due to platform tags. Review Tests tab for skipped counts.
Was this page helpful?