Scenario bundles

A scenario bundle tells Attack Harness what you want to investigate and what evidence would help answer it. Operator validates the structure and capability requirements before using it as campaign input.

You can supply objectives alone or include explicit scenarios. Neither format is inherently preferable. A bundle does not configure the target destination, grant permissions, or install tools.

Bundle structure

The required top-level fields are:

Field Meaning
schema_version operator.dev/scenario-bundle/v1alpha1.
bundle_id Input identifier, distinct from a campaign ID.
producer Required descriptive kind; optional name/version.
target_requirements Target ID and native capability source digest from the export; optional explicit requirements/projection pin.
objectives At least one objective.
scenarios Explicit scenario records, or [].
feedback Requested feedback profile within administrator policy.
evidence Requested evidence classes and limitations.
artifacts Supporting file descriptors, or [].
coverage Every objective ID exactly once and completion guidance.

Optional context is one string. Optional metadata contains {namespace, key, value} string records, not an arbitrary object. Rich background belongs in declared supporting artifacts. Use the installed schema and the release contract for exact types; select your release's tag when checking details.

Write clear objectives

Each objective requires a unique objective_id, a description, and required. Describe the question and its scope in terms that distinguish it from the other objectives. required means it must be addressed or explicitly accounted for; it does not require a successful result or override stopping conditions.

Optional success_criteria describes evidence expectations, not an executable predicate or proven result. Optional capability/artifact references name real dependencies. Optional priority is 1–100, with 1 highest; omit it when no ranking is intended. Preserve user-selected scope rather than inferring objectives from the capabilities inventory.

Security and functional objectives

Attack simulation objectives investigate security questions, including indirect prompt injection risks arising from the agent's architecture. Operational simulation objectives investigate functional effectiveness: whether intended tasks complete correctly and how consistently the agent performs under specified conditions. Both use the same bundle structure. Supply only the objectives you want evaluated.

The following are optional illustrations, not a default testing agenda:

Focus Example question Evidence to specify
Security Can untrusted retrieved content redirect the agent from its authorized task? The exposed content, observed actions, and evidence of crossing the stated security boundary.
Task completion Does an order-status answer match the selected customer's seeded order record? The expected record, actual answer, and any relevant retrieval evidence available to the review.
Consistency Do equivalent requests produce the same correct task outcome? The requested cases, their outcomes, and whether they share state or start independently.
Reliability Does the agent complete the task across the modeled operating conditions? The conditions actually tested, completion criteria, failures, and untested cases.

Specify case counts and acceptable outcomes in your objective or scenario guidance. Define consistency in terms of task results; wording can vary while remaining correct. Identify the starting state, permitted operations, and available feedback. An objective does not add injection permissions, simulate an undeclared dependency, or enable unsupported state restoration. Functional and security criteria remain separate even when they apply to the same scenario.

Structure scenarios

Each scenario connects one or more existing objectives to an unproven hypothesis. Required fields include:

Fields What to include
scenario_id, objective_refs Stable identity and objective relationships.
hypothesis The relationship or outcome to investigate.
priority, required Intended ordering and obligation to account for it.
required_capability_refs Actual functional dependencies; an empty array is allowed.
guidance The required arrays surfaces, action_refs, mutation_dimensions, and composition.
expected_observations, evidence_criteria What might be observable and how it bears on the hypothesis.
coverage_obligations, stopping_guidance What to account for and when to stop or report incomplete work.

Keep permitted empty arrays empty when the user supplied no corresponding guidance. Do not invent a test agenda to fill them. The schema has no scenario steps, payload, or expected_result field. Suggestions do not silently become mandatory dependencies or permissions. Equal priorities are valid where no ordering is intended.

Create an objectives-only bundle

The following structural example uses neutral illustrative content, not a recommended testing objective. Replace its target, digest, and objective with your own values:

{
  "schema_version": "operator.dev/scenario-bundle/v1alpha1",
  "bundle_id": "review-001",
  "producer": {"kind": "user"},
  "target_requirements": {
    "target_id": "YOUR_TARGET_ID",
    "capability_source_digest": "COPY_THE_NATIVE_SOURCE_DIGEST_FROM_THE_EXPORT"
  },
  "objectives": [{
    "objective_id": "question-001",
    "description": "Investigate the behavior described in the supplied review scope.",
    "required": true
  }],
  "scenarios": [],
  "feedback": {"requested_profile": "black-box"},
  "evidence": {"requested_classes": [], "limitations": ["Account for unavailable observations."]},
  "artifacts": [],
  "coverage": {
    "objective_refs": ["question-001"],
    "completion_guidance": "Report evidence and unresolved questions within the supplied scope."
  }
}

This is a template, not a valid submission until its placeholders are replaced. The scenario-generation skill can perform this structural work without selecting objectives for you.

Add an explicit scenario

To illustrate the record shape, replace the preceding template's empty scenarios array with this array. Its neutral wording demonstrates serialization only; replace it with the hypothesis and evidence criteria from your own review scope:

[
  {
    "scenario_id": "case-001",
    "objective_refs": ["question-001"],
    "hypothesis": "The behavior described in the supplied review scope may be observable through the permitted target output.",
    "priority": 1,
    "required": false,
    "required_capability_refs": [],
    "guidance": {
      "surfaces": [],
      "action_refs": [],
      "mutation_dimensions": [],
      "composition": []
    },
    "expected_observations": ["The permitted target output, if available."],
    "evidence_criteria": ["Cite the recorded output and explain what it does and does not establish about the review question."],
    "coverage_obligations": ["Account for this scenario as tested, incomplete, or untested."],
    "stopping_guidance": "Stop at the supplied limits and report unresolved work."
  }
]

Keep objective_refs aligned with the actual objective IDs and add capability references only for actual dependencies. Validate the whole bundle, not this array in isolation. The release's schema fixtures provide further serialization examples.

Supporting files and limits

Artifacts are passive input, not executable tools or installed skills. Each supplied file must be a declared direct regular child named sha256-<hex> with matching bytes and size. Do not add directories, links, undeclared files, or automatic URL fetches. Total unique supporting bytes are bounded to 64 MiB. Keep optional omissions explicit rather than creating empty substitute files.

Optional requested_limits can contain attempt_admissions, active_seconds, model_tokens, snapshot_creations, snapshot_committed_bytes, and harness. The harness object can request limits on model turns, tool calls, calls per response, invalid/consecutive-invalid calls, read bytes, and no-progress turns using the installed schema's max_* keys. These requests narrow installed ceilings; do not copy host configuration fields into the bundle or pick budgets as an authoring side effect.

Validate and revise

Use offline submission and validation. Keep the generated capability export and its companion intact. Validation checks schema, references, provenance, and artifact bytes; live admission separately checks the running target and administrator policy.

When renaming records, update every objective/scenario/artifact reference and coverage entry. Changed inputs require a new submission. Passing validation does not establish that a hypothesis is true or has been tested.