Benchmark Methodology
Every Zof benchmark discloses environment, workload, sample size, variance, and limitations before any result is published.
Benchmarks that cannot be reproduced should not influence buying decisions. We publish methodology first and label framework pages clearly when data is in progress.
What this suite tracks
checklist
Methodology disclosure completeness
Required fields present before publication.
checklist
Independent reproducibility
Third party can replay using published artifacts.
checklist
Governance and safety scoring
Policy adherence measured where remediation is involved.
How we measure
Benchmark suites align to Zof product pillars: testing fleets, remediation fleets, System Graph, deployment planes, and reliability ROI. Comparison sections describe capability dimensions, not hostile competitor claims.
| Test environment | Documented reference stacks per suite (web, API, workers, CI, observability). Customer topologies mapped during assessment, not assumed. |
|---|---|
| Dataset / workload | Versioned scenario packs with ground-truth labels where accuracy is scored. |
| Sample size | Declared minimum per suite; runs below minimum are not aggregated. |
| Number of runs | Declared run count with warm/cold separation where latency matters. |
| Variance | Future published results include p50/p95 and dispersion, not single runs cherry-picked. |
| Excluded runs | Infrastructure failures, connector outages, and policy violations excluded and counted. |
| Date last run | Pending first benchmark run |
| Version tested | Pending first benchmark run |
| Repeatability | Artifact packs, scenario YAML, and runner manifests publish alongside results. Framework-only pages link here and state “results pending.” |
Assumptions
- -No competitor data unless sourced from public materials with date stamps.
- -No customer quotes without written approval.
- -Framework pages never display unsupported percentages.
Results pending first benchmark run
This page does not display performance numbers until completed runs pass validation. When published, results include confidence ranges and sample sizes.
| Metric | Value | Confidence range | Notes |
|---|---|---|---|
| Methodology disclosure completeness | Pending | - | Awaiting completed runs |
| Independent reproducibility | Pending | - | Awaiting completed runs |
| Governance and safety scoring | Pending | - | Awaiting completed runs |
What this benchmark does not claim
- -Reference environments simplify real-world complexity.
- -Customer results require separate approval and may not match aggregate suites.
- -Comparison tables describe architectural fit; they are not independent third-party audits.
Enterprise interpretation
Treat framework pages as evaluation rubrics. Engage Zof architects to map suites to your topology before relying on any published numbers.
Continue your evaluation
Evaluate Zof against your reliability requirements
Review methodology, run a structured assessment, or benchmark against your workflow with enterprise architects.
