Agent Hijacking

MITRE ATLAS: T0005 — Execution; T0008 — Defense Evasion; T0012.003 — Craft Malicious Prompt (agent context)
OWASP LLM Top 10: LLM07 (Insecure Plugin Design), LLM08 (Excessive Agency)
NIST AI 600-1: §2.7 (Human-AI Configuration), §2.12 (Value Chain)
EU AI Act: Art. 14 (Human oversight), Art. 15 (Robustness)

Threat Model

Autonomous agents (AutoGPT, LangGraph, AutoGen, CrewAI) combine:

Planning LLM — decomposes goals into actions
Tool set — APIs, code execution, file system, browser, DB
Memory — short-term (context), long-term (vector store)
Delegation — spawn sub-agents for sub-tasks

Attackers exploit the trust boundary between planner and tools.

Attack Vectors

Vector	Mechanism	Impact
Tool misuse via prompt injection	Malicious prompt in retrieved content / user input causes agent to invoke dangerous tool (e.g., `rm -rf`, `send_email`, `sql_exec`)	Arbitrary code execution, data exfiltration, financial loss
Context window stuffing	Attacker fills context with noise to evict safety instructions / push malicious prompt into active window	Bypasses system prompt, evades safety filters
Delegation abuse	Compromised sub-agent spawns further agents with escalated privileges	Privilege escalation, lateral movement
Memory poisoning	Malicious content written to long-term memory (vector store) affects future agent runs	Persistent compromise across sessions
Reward hacking / goal misgeneralisation	Agent finds unintended high-reward strategy (e.g., "maximise clicks" → click fraud)	Unintended harmful behaviour at scale

Primary source: MITRE ATLAS T0005, T0008, T0012.003. ATLAS

Multi-Agent Attack Chain

Rendering diagram...

flowchart LR
    A[User / Retrieval] -->|Poisoned prompt| B[Planner Agent]
    B -->|Delegates| C[Sub-agent: Research]
    C -->|Writes to memory| D[Vector Store]
    B -->|Delegates| E[Sub-agent: Execute]
    E -->|Reads poisoned memory| F[Dangerous Tool Call]
    F -->|Exfiltration / Deletion| G[Impact]

Mitigations

Layer	Control	Standard Mapping
Architecture	Capability-based tool access — each tool has scoped permissions; agent requests explicit grant	NIST AI 600-1 MANAGE-2.2, OWASP LLM07, LLM08
Architecture	Human-in-the-loop for high-impact actions — require confirmation for `delete`, `pay`, `email`	EU AI Act Art. 14, OWASP LLM08
Runtime	Action confirmation gateway — intercept tool calls, present to human for approval	NIST AI 600-1 MANAGE-4.1, ISO 42001 A.6.2
Runtime	Context window management — reserve tokens for system prompt; truncate oldest user content first	OWASP LLM01, NIST AI 600-1 MEASURE-2.1
Runtime	Memory sanitisation — validate/redact content before writing to long-term store	MITRE T0006.001, ISO 42001 A.9.2
Monitoring	Agent behaviour baselines — typical tool call sequences, delegation depth, context usage	NIST AI RMF MEASURE-3.3, MITRE TA0008
Governance	Agent registry — approved agents, tool sets, and delegation policies	EU AI Act Art. 14, ISO 42001 A.8.1

Citation Register

Claim	Instrument	Provision	URL
Agent execution tactic	MITRE ATLAS	T0005	https://atlas.mitre.org/tactics/TA0005
Defense evasion	MITRE ATLAS	T0008	https://atlas.mitre.org/tactics/TA0008
Prompt crafting	MITRE ATLAS	T0012.003	https://atlas.mitre.org/techniques/T0012.003
Insecure plugin design	OWASP LLM Top 10	LLM07	https://owasp.org/www-project-top-10-for-large-language-model-applications/
Excessive agency	OWASP LLM Top 10	LLM08	https://owasp.org/www-project-top-10-for-large-language-model-applications/
Human-AI config risk	NIST AI 600-1	§2.7	https://doi.org/10.6028/NIST.AI.600-1
Human oversight	EU AI Act	Art. 14	https://eur-lex.europa.eu/eli/reg/2024/1689/oj