Functional anatomy
Real systems are not clean flowcharts. Stages overlap, evidence arrives late, confidence decays, communications fail, human attention is limited, and feedback can send the process backward or trigger a new review.
Answer-first summary
Sensors produce observations; systems and people establish identity and location, maintain tracks, validate and prioritize a course of action, apply an authorized effect, and assess the result.
Read the supporting sectionIt organizes an intrusion from reconnaissance through weaponization, delivery, exploitation, installation, command and control, and actions on objectives.
Read the supporting sectionThey can manipulate data, context, models, memory, tools, and downstream actions without requiring a conventional malware foothold.
Read the supporting sectionMilitary F2T2EA
F2T2EA is useful because it exposes distinct evidence, authority, and failure questions at each stage. An interruption can be physical, informational, cyber, electronic, legal, procedural, or human.
01 · Sensing
A synthetic sensor network detects an object inside a closed training environment.
AI may help scan large sensor feeds and highlight anomalies, but a detection is only a cue. It is not identity, legal status, or authorization to act.
Synthetic wide-area sensor cue with source timestamp and confidence interval.
Is the observation relevant, current, and from a trusted sensor path?
02 · Identity & location confidence
Independent synthetic sources are correlated to refine identity and position uncertainty.
A classifier score is not positive identification. Provenance, contradictory observations, uncertainty, and stale data must remain visible.
Two simulated sources agree; one lower-confidence source conflicts.
What evidence supports the identity, and what evidence contradicts it?
03 · Continuity
The synthetic system maintains custody while displaying uncertainty growth during sensor loss.
Predictive tracking can bridge short gaps, but the interface must distinguish observation from inference and decay confidence over time.
Track continuity with a visible gap and model-predicted segment.
Has the system preserved identity, or merely continued a plausible trajectory?
04 · Decision support
Humans review mission purpose, evidence, alternatives, constraints, and legal requirements.
AI may rank or recommend, but legal judgment, proportionality, rules of engagement, and command responsibility cannot be reduced to a model score.
A synthetic course-of-action card with uncertainty, provenance, alternatives, and no-strike constraints.
Is the proposed action lawful, necessary, proportionate, and supported by sufficient evidence?
05 · Authorized simulated effect
A human-authorized, non-destructive simulated effect is applied inside the training environment.
The model represents authority boundaries, authentication, abort paths, and safe-state behavior—not weapon selection, trajectories, yields, or target coordinates.
Signed synthetic authorization, bounded time window, independent abort channel, and safe-state monitor.
Who authorized the effect, what remains reversible, and can the system be stopped independently?
06 · Effects & accountability
Post-action evidence is compared with the intended outcome without automatically generating a repeat action.
Assessment models can be wrong. Re-engagement should never be an unreviewed consequence of an uncertain machine assessment.
Synthetic before/after evidence, confidence, residual uncertainty, and audit trail.
What is known, what remains unknown, and does any follow-on action require a new decision?
What the software stack is doing
A modern chain is a system of systems. The most safety-critical error may originate upstream in a timestamp, identifier, data transformation, or interface—not in the final seeker.
Layer 1
Radar, electro-optical, infrared, RF, acoustic, cyber, and space observations are calibrated, stabilized, denoised, time-aligned, and georegistered.
Layer 2
Tracks are associated, deduplicated, correlated, aged, and assigned uncertainty. Learned models may complement physics-based filters.
Layer 3
Targets or threats may be ranked, courses of action generated, resources allocated, and sensor–effector pairings recommended.
Layer 4
Authentication, mission bounds, legal constraints, human authorization, runtime assurance, abort paths, and safe states gate external effects.
Layer 5
Post-action evidence updates the operational picture, but it should not automatically generate another engagement. Software, model, mission, sensor, or environmental changes may require revalidation.
Human role transformation
The human may leave continuous control without leaving the decision architecture. A rigorous explanation identifies which function moved, which authority remained human, when intervention was possible, and which evidence the human actually received.
| Function moved to software | Why organizations automate it | Human role that remains | Main control risk |
|---|---|---|---|
| Observation filtering | Data volume exceeds human review capacity | Set evidence rules, inspect provenance, audit what was suppressed | The algorithm controls the visible choice set |
| Track correlation and fusion | Many asynchronous observations must be reconciled continuously | Review contradictions, source independence, data age, and uncertainty | A polished composite track can hide weak or circular evidence |
| Ranking and recommendation | Concurrent cases exceed available attention and time | Question priorities, reject defaults, request corroboration, preserve Unknown | Human approval becomes a rubber stamp |
| Navigation and task execution | Continuous teleoperation does not scale and links may fail | Define mission intent, boundaries, safe states, expiration, and exceptions | Undefined authority migrates onboard during silence |
| Bounded local defense | The response window may be shorter than a meaningful manual sequence | Configure, activate, supervise, abort where practical, investigate afterward | Earlier human assumptions are executed consistently at machine speed |
| Assessment and re-tasking | New observations must update the picture continuously | Review uncertainty and decide whether a new action requires fresh authorization | An erroneous assessment creates an automatic repeat loop |
Classic Cyber Kill Chain
The seven-stage model explains campaign progression and defender interruption. MITRE ATT&CK provides behavior-level tactics and techniques that can occur concurrently, repeat, or map to several narrative stages.
01 · Observe
A fictional actor gathers public information about a synthetic organization.
The simulator shows exposed information and defender-visible signals without providing scanning or collection instructions.
02 · Prepare
A nonfunctional abstract capability and lure are assembled.
No executable artifact is created. The focus is infrastructure preparation, impersonation signals, and pre-delivery detection opportunities.
03 · Cross a boundary
A harmless synthetic message reaches a training inbox.
The message carries only declarative scenario data. It cannot contact external systems or execute content.
04 · Trigger a weakness
A fictional control failure changes a host state inside the simulation.
The event is represented as a semantic state transition, not a functional exploit or command.
05 · Persist
A non-executable persistence marker appears on a synthetic endpoint.
Learners inspect persistence evidence and choose isolation or restoration controls without handling malware.
06 · Coordinate
A fictional endpoint emits a beacon-like pattern to a reserved domain.
The destination is non-routable and the simulator provides no C2 implementation details.
07 · Impact
The synthetic scenario models attempted data access and service disruption as abstract effects.
No data is collected and no system is modified. The learner focuses on blast radius, recovery, notification, and lessons learned.
Attack chain against AI
An image can induce misclassification; queries can reveal a model; a poisoned document can redirect an agent; a compromised model artifact can execute ordinary software attacks. AI-specific and conventional cybersecurity models are both required.
01 · Map the trust boundary
A synthetic adversary probes exposed behaviors, data routes, tools, and error messages.
The attack surface includes every document, tool result, memory item, model endpoint, and integration that can enter the model context.
02 · Place malicious data
A synthetic untrusted document enters a retrieval corpus with instruction-like content.
“Poison” is broader than training-data poisoning: it can include prompts, external pages, tool metadata, shared memory, or retrieved documents.
03 · Redirect model-mediated behavior
The synthetic model treats retrieved text as an instruction and proposes an unauthorized tool action.
The security boundary must not depend on a probabilistic model correctly distinguishing instruction from data on every input.
04 · Write poisoned state
The synthetic agent attempts to store the instruction in shared memory for later sessions.
Persistence can live in memory, retrieval indexes, summaries, tool configuration, or agent plans—not only in model weights.
05 · External state change
A synthetic tool call would create an unauthorized external action, but the simulator contains it.
Autonomy is an impact multiplier. A read-only assistant may produce a bad answer; an over-privileged agent can change systems, communicate, or expose data.
Why chains loop
Linear diagrams remain useful for teaching, but real chains can branch, merge, retry, skip stages, or run several paths at once.
Assessment may reveal that the intended effect was not achieved, that identity changed, or that the operational context is different. A responsible process returns to evidence and authorization rather than treating “assess” as an automatic re-engagement trigger.
A hijacked agent may discover tools, write persistent state, retrieve new instructions, or propagate across users and services. Each new capability expands the trust boundary and potential impact.
An adversary may change delivery paths, re-enter reconnaissance after a failure, maintain several command channels, or pursue different objectives after gaining access.
Distributed networks can replace lost nodes and select alternate sensors or effectors. The same resilience can accelerate the propagation of a false track unless source lineage and authority remain explicit.
Design principle
The strongest architecture lets an AI propose a route, classification, pairing, or tool call while an independent runtime-assurance layer enforces authentication, target or resource bounds, time and geography constraints, data provenance, sensor-health requirements, rate limits, and safe-state transitions.
Human override should not depend on the same model, processor, credential, or communications path that may be compromised. In AI applications, deterministic authorization must remain outside the language model. In military systems, legal judgment and command responsibility remain human obligations.
Machine-mediated institutional chain
A common institutional sequence is: human mandate → machine interpretation → machine prioritization → machine planning → machine coordination → automated routine execution → human exception review → public or corporate consequence → human accountability.
Control is strongest when the objective, source data, proxies, thresholds, permissions, agent handoffs, overrides, corrections, and final responsibility remain visible and technically enforceable.