Simulation Replay Acceptance: Which Clock Drives the Result?

A simulation supplier delivers a recording, a scene and a short video. All three open, yet the buyer cannot tell which clock determined the reported sequence of events. This is a useful acceptance question because a playable demonstration and a repeatable analysis are different deliverables. The following review concerns an illustrative software package, not a test we conducted. It proposes questions a buyer can put to the supplier before recording the simulation work as accepted.

Specify what the replay is supposed to reproduce

Start with the promised observation. Does the buyer need to inspect a recorded sensor stream, rerun an algorithm against that stream, or recreate a simulated interaction? Those tasks may share files without sharing an acceptance criterion. A video can help explain a scene, but it does not by itself identify the messages consumed by a particular run. Ask the supplier to name the intended task and the particular output that a reviewer should inspect.

For the hypothetical delivery, suppose the agreed task is to review the ordering of recorded events. The acceptance record should identify the recording, the consumer configuration and the output being examined. It should also state what the exercise excludes. Reproducing an ordering does not automatically reproduce numerical results from an unspecified algorithm, nor establish that a simulated robot behaves like physical equipment. Keep those additional claims outside the conclusion until they receive their own evidence.

Keep three timing questions on the review sheet

The first question concerns timestamps stored with the recording. The second concerns the time source used by the receiving application. The third concerns how the playback session progresses. A reviewer should request a separate explanation for each, even if the supplier says that all components use simulation time. That phrase is too broad to identify the relevant configuration or demonstrate that the intended consumer actually used it.

The ROS Clock and Time design document describes ROS time becoming active through the node's use_sim_time setting and discusses abstracted time that can pause or jump. This is a design reference, not evidence about the delivered program. Its practical use here is to frame a question: which components use the abstracted clock, and what records support that answer for the supplied version?

Ask what happened before the clock became available

The current rolling rosbag2 project documentation explains that recording with its simulated-time option uses received clock values for recorded timestamps and waits for the first clock message before writing messages. This is a specific documented behavior, not a universal rule for every recording tool. Ask for the delivered tool's version and applicable documentation rather than assuming a different release follows every detail of the current branch.

The supplier's answer should distinguish an expected initial waiting period from a missing portion of the agreed recording. Request an explanation of the intended start boundary and how a reviewer can locate it. Do not invent a duration from the video's first frame or treat an empty opening display as proof of lost data. If the package does not contain enough information to decide, record the start boundary as unresolved and request the missing explanation.

Turn a restart into a defined review case

A replay that returns to an earlier recorded position raises a different question from one that merely runs more slowly. Ask how the supplied analysis handles that case, if it belongs to the contract. The reviewer needs to know what state is retained, what is reset and how the supplier expects an earlier event to be interpreted. This article does not prescribe implementation changes or suggest trying them on connected machinery.

Arrange any demonstration in an authorized, isolated software environment with the supplier. Before it begins, identify the starting package and the observation to collect. Afterwards, keep the reported result attached to that exact case. A successful replay from one starting point does not answer every restart or time-jump question. Conversely, an undocumented restart case should not be described as a failure of an experiment nobody performed.

Write an acceptance conclusion another reviewer can repeat

A useful conclusion names the task, package, timing assumptions and observations actually reviewed. Separate files received from behavior demonstrated, and separate behavior demonstrated from broader reproducibility claims. If only the recording has been inspected, say so. If configuration evidence remains missing, identify the component and clock question that still need an answer. This produces a bounded decision rather than a general statement that the simulation is reproducible.

Keep the supplier's version-specific instructions with the review, including any stated limitations on replay. The drawings here are editorial maps of questions, not timing measurements or a product architecture. Related discussion in our simulation reading collection covers other reproducibility concerns; the article directory provides the wider topic list. Neither a readable recording nor an attractive simulation image substitutes for the particular evidence promised at procurement.