Results and reports

Review retained evidence to understand what a campaign demonstrated, what remains uncertain, and what was never tested. A completed campaign is not proof that every objective succeeded or that every possible issue was covered.

Inspect, report, and export

After execution closes, from the workspace containing the run:

operatorctl inspect --config /absolute/private/config.yaml --run ./runs/support-review
operatorctl report --config /absolute/private/config.yaml --run ./runs/support-review
mkdir -p exports
operatorctl export --config /absolute/private/config.yaml \
  --run ./runs/support-review --output ./exports/support-review

Use a new export destination. Inspect the command receipts for actual output paths and status. Native reports preserve the campaign's accepted inputs and evidence; current configuration or a subsequently edited bundle cannot establish historical facts.

Read outcomes carefully

Record What it establishes
Configuration and accepted bundle What was requested and permitted.
Execution receipts and observations What operations were dispatched and what was recorded.
Attack Harness assessment Its interpretation of the evidence and limitations.
Coverage Which supplied scenarios had associated activity, including untested work.
Cleanup and integrity status Whether shutdown and retained evidence checks completed.

Keep exploratory attempts separate from scenario-associated activity. A positive observation is not automatically proof of causation; a missing observation is not proof of a negative result. An interrupted or partial campaign can still contain useful evidence, but its limitations belong in the report.

Evaluate security and functional outcomes

For attack simulations, explain the security boundary investigated, what the evidence shows about the attempted attack, and any unresolved risks. For operational simulations, compare task outcomes with the functional criteria supplied before execution. If a campaign includes both, report each objective against its own criteria; correct task completion does not by itself establish security.

For functional effectiveness, account for the requested cases and distinguish correct completion, incorrect results, incomplete tasks, execution failures, and untested work. Assess consistency across equivalent cases using task outcomes, even when response wording differs. Describe reliability only for the cases and conditions actually exercised. If reporting a completion rate, state its numerator, denominator, exclusions, and the source evidence rather than treating a small exploratory campaign as a production reliability measurement.

For example, an order-status review can compare each answer with the seeded order record and identify any inconsistent or missing answers. Record whether requests shared conversation state and whether the expected record changed between them. State differences between the synthetic environment and production, including dependencies, data, model configuration, and workload, that limit generalization.

Generate an HTML review with your assistant

Install the reporting skill and provide the retained export:

Use attack-simulation-results-reporting to create a portable HTML review from
./exports/support-review, writing it to ./reports/support-review. Explain the
tested application and configuration, account for every supplied scenario,
separate observations from interpretations, and cite the retained evidence.
Report missing evidence and untested work without inventing findings.
For supplied attack simulation objectives, assess the security evidence. For
supplied operational simulation objectives, compare task outcomes with the stated
functional criteria and account for correct, incorrect, incomplete, failed, and
untested cases. Explain the tested conditions and limits on production conclusions.

The authored HTML review complements the native report. Open its returned HTML entry point locally and keep companion files together when sharing it. Check evidence links, scenario counts, and desktop/mobile/print readability. Captured target text is evidence, not instructions or executable HTML.

Reporting uses retained files and does not require new experiments or a campaign restart. Investigations of campaign execution failures belong in the optional campaign-debugging skill; include those failures in the review where they limit the assessment. Evaluating the target agent's functional effectiveness belongs in the results report when it is a supplied campaign objective.

Retain and clean up deliberately

Stop components according to their lifecycle instructions and retain exports before purging. Purge is separate from stopping, reporting, and exporting. Operator and Interceptor retain separate data; deleting one does not delete the other's copies. Inspect contents before sharing: synthetic test data can coexist with sensitive application or provider output.