Skip to content

Workflow evaluation

After the plant connection returns, is the order still in the right state?

Connectivity can recover before the business state does. Follow a held manufacturing order through delayed messages and repeated delivery to see what a release review should establish.

6 minute read By Zof Updated October 26, 2026

This guide presents an evaluation method, not a report of a completed customer engagement.

Workflow
Order and quality state exchanged between ERP and MES
Change
An interface update affecting queued event recovery
Constraint
Representative software testing outside live plant control
Decision
Whether recovery evidence supports the planned release

A connection drops while a manufacturing order is being updated. A quality hold is recorded before the link returns. When queued messages arrive, an older release instruction follows the hold. Every message was delivered, but the final state may still be wrong.

Evaluate the software interface between an enterprise resource planning system and a manufacturing execution system using isolated test orders and events. Keep live machinery and safety functions outside the exercise.

Define which system owns the quality decision

Assume the MES owns the quality hold for the order, while ERP owns its planning information. The interface distributes updates between them. For this evaluation, a hold remains effective until an authorized newer quality decision removes it. An older release event must not override that decision simply because it arrives later.

These state ownership assumptions must be checked against the actual ERP and MES contracts. The plant team must supply the actual ownership, version and transition rules. Without that agreement, a test can show two different final states but cannot establish which one is correct.

A successful reconnect says little about recovery

A connectivity check can confirm that messages move again. It does not establish that the receiver applies them in a valid order, recognizes a repeated delivery or preserves a current hold. A normal sequence with one message per transition leaves those questions unanswered.

The release review therefore follows one order through interruption and recovery. It inspects the business state after each relevant event, not just the transport acknowledgment. The key question is whether an old or repeated event can create a transition that the current quality decision forbids.

Follow the workflow

A quality hold must survive the delivery order

  1. 01

    Planning interface

    A release event is queued

    The synthetic order has a release event at revision two. Connectivity fails before the receiver finishes processing it.

  2. 02

    MES quality service

    A newer hold is recorded

    The quality owner records a hold at revision three. The source retains the decision while the interface remains disconnected.

  3. 03

    Recovery path

    Messages arrive in a different order

    The hold reaches the receiver first. The older release event follows, and a copy of the hold is delivered again.

  4. 04

    Receiving application

    Current state remains held

    The application applies the agreed revision and duplicate rules. It records the old and repeated events without releasing the held order.

  5. 05

    Release owner

    Inspect the whole transition

    Compare source authority, event identities and final state. Review any gap between the test configuration and the intended plant deployment.

Recovery sequence for the defined MES quality ownership model. Business revisions establish order; arrival time does not grant authority to an older event.

Make the test boundary explicit before execution

Use an isolated software environment with synthetic orders and no route that can issue commands to live equipment. Match the relevant interface versions, queue settings and state rules to the intended release. Document any differences in timing, storage or connectivity that could affect recovery.

NIST guidance for operational technology security recognizes the distinct performance, reliability and safety needs of operational systems. Here, that supports a carefully bounded software exercise. It does not make a passing interface test a safety assessment or proof that a production line is ready.

Connect the changed interface to the state it can affect

A scoped Zof evaluation would use the System Graph to connect the interface change to order events, the quality decision and the receiving application. Plant engineers confirm that map and identify which relationships matter to the planned update.

The Verification Engine would exercise the agreed interruption, repeat and ordering cases. Select edge execution or deployment on premises only where it fits the approved environment. The test setup must provide the required event controls; the scenario assumes no particular vendor adapter, digital twin or automatic offline buffering feature.

An evaluation you can inspect

Recovery checks for the order interface

Each recovery check uses the same test order and explicit state ownership. Agree the timing and replay controls for the test environment.

Recovery checks for the order interface. Checks and acceptance criteria.
Recovery conditionRequired behaviorEvidence to inspect
Events arrive in business orderApply the release, then the newer hold; finish in the held state.Source revisions, delivery sequence and final quality state.
Older release arrives after the holdPreserve the newer hold and reject the stale state transition.Recorded hold revision and handling of the older release event.
Hold event is delivered twicePreserve one effective hold without an unintended repeated transition.Both delivery attempts, duplicate identity and resulting state.
Link returns but an event is missingExpose incomplete reconciliation under the agreed rule.Known source revision, received events and the unresolved gap.
Consumer restarts during recoveryRetain the agreed recovery state and duplicate handling across restart.Persisted state before restart, replay attempts and final reconciliation.

On a small screen, scroll the matrix to compare all columns.

Download the evaluation worksheet

Failure condition to test

Failure condition: a stale event removes a current hold

Condition to exercise
Deliver an older release event after the newer quality hold. Check whether the receiver applies arrival order and incorrectly changes the order to released.
Required behavior
The newer authoritative hold remains effective until an authorized quality decision removes it. Late arrival does not make the older release event current.
Evidence to collect
  • The agreed ownership rule and synthetic source revisions.
  • The hold, late release and repeated hold in delivery order.
  • Receiver state before and after each event, with the consumer version.
Release decision
Keep release approval open for this path. Review the state transition logic, reproduce the failure and rerun the matrix after an approved change.

Make the release decision from the recovery record

For each case, inspect the order before interruption, the authoritative quality revision, the delivered events and the state after recovery. Record event identity and business version as well as timestamps. Clocks alone cannot establish authority when systems are disconnected or their time differs.

The acceptance condition is that the hold remains effective, order state reconciles under the agreed ownership rules and repeated events do not create an unintended second transition. A contradictory record, missing trace or unexplained state change keeps the case open. Correct transport behavior cannot compensate for incorrect business state.

Know what the evidence cannot carry into production

A test against a different queue policy or consumer version cannot settle the production question. Record those differences in the release review and decide whether another check is necessary. A restart during recovery is a separate condition if it was not exercised in the agreed scope.

For a manufacturing software team, the output is a reproducible recovery record and a list of exceptions the release owner can act on. Measure operational changes such as downtime or throughput separately, against a defined baseline and observation period after deployment.

Before you scope the work

Will this exercise touch a live production line?

Use an isolated software environment for this evaluation. Any production activity needs a separate explicit scope, plant authorization and operational controls. A software recovery test is not permission to exercise physical equipment.

What must match the plant configuration?

Prioritize the interface and consumer versions, state rules, delivery policy and persistence behavior. Record material differences. A representative environment is a documented comparison, not a label applied to a lab.

Who decides whether the order is correct?

The process owner and plant engineering team define the state authority and permitted transitions. Zof evaluates behavior against those agreed criteria; it does not invent the quality policy.

What if an event cannot be reproduced safely?

Mark that condition as untested and explain its consequence for the release decision. Agree a safer simulation or narrower claim. Do not count an unavailable test as a passing recovery check.

Recovery is a state decision, not a connection indicator

A restored link matters only if the receiving application preserves the decisions already made. Follow the order, its authority and its final state before accepting the software change.

Sources and scope

These primary references explain the mechanisms and review practices used in this evaluation.

  1. Guide to Operational Technology Security, revision 3 · NIST

    Guidance recognizing operational technology performance, reliability and safety requirements. It informs the test boundary; event semantics must come from the system’s actual contract.

Your workflow. Your evidence.

Define the evidence for one plant software workflow

Bring the interface owner, a representative test environment and the release decision. A paid Reliability Sprint scopes one system, typically over seven days, with access, operational boundaries and price agreed before testing.

Book a Reliability Sprint

A paid engagement for one critical system. Scope and price are agreed first. Seven days is typical; access and deployment requirements can extend the timeline.

Continue the investigation

Retail and payments

When a payment times out, can checkout safely retry?

Compare recovery with payment retries, where repeated delivery can also change business state.

Systems integration

Reuse the acceptance method. Keep each client's evidence separate.

See how an acceptance method must be bound to the configuration of each client environment.

01Zof Console

One surface for posture, operations, and what needs attention next.

The authenticated home that engineering, QA, and SRE teams open every day: quality posture, in-flight runs, coverage by module, and what needs attention next.

OPERATIONAL KPIs

  • Runs
  • Coverage
  • Risk

Live across every environment you ship to.

WORK SPINE

  • Specs
  • Tests
  • Schedules

From specification to scheduled regression.

GUARDRAILS

  • RBAC
  • SSO
  • audit

Every action attributable to a named human.

LIVE/console
Zof home command center showing 12 runs at 94% pass, 3 open critical issues, 84% coverage, four module traceability bars, the specification pipeline, upcoming schedules, and recommended next actions with an active-runs sidebar.
Example console home · Checkout Service · Staging.