Adversarial Attacks

MITRE ATLAS: T0012.001 — Craft Adversarial Examples
OWASP LLM Top 10: LLM01 (adversarial prompts), LLM04 (DoS via complex adversarial inputs)
NIST AI 600-1: §2.1 (CBRN), §2.2 (Confabulation), §2.3 (Dangerous Content)
EU AI Act: Art. 15 (Accuracy, robustness, cybersecurity)

Attack Classes

Class	Access	Method	Key Papers
GCG (Greedy Coordinate Gradient)	White-box (gradients)	Optimises discrete token substitutions via coordinate descent on loss	Zou et al. 2023, "Universal and Transferable Adversarial Attacks on Aligned Language Models"
PAIR (Prompt Automatic Iterative Refinement)	Black-box (API)	Attacker LLM iteratively refines jailbreak via judge LLM feedback	Chao et al. 2023, "Jailbreaking Black Box LLMs"
TAP (Tree of Attacks with Pruning)	Black-box (API)	Monte Carlo tree search over prompt variations with pruning	Mehrotra et al. 2024, "Tree of Attacks with Pruning"
Gradient-based (white-box)	White-box (full model)	Direct gradient descent on embedding space (PGD, AutoPrompt)	Wallace et al. 2019, "Universal Adversarial Triggers"
Transfer attacks	Black-box (surrogate)	Optimise on open-source surrogate, transfer to target	Zou et al. 2023 (transferability of GCG suffixes)

Primary source: MITRE ATLAS T0012.001. ATLAS

GCG (Greedy Coordinate Gradient)

Initialise adversarial suffix (e.g., 20 random tokens)
Compute gradients of loss w.r.t. one-hot token embeddings
Coordinate descent — for each position, evaluate top-k token substitutions by gradient dot product
Accept substitution reducing loss most; repeat until convergence
Result — short suffix (e.g., 20 tokens) that, when appended, causes refusal bypass

Transferability: GCG suffixes optimised on Llama-2 transfer to GPT-3.5/4, Claude, PaLM at non-trivial rates (~30–60% ASR).

Attacker LLM ←→ Judge LLM
    │                │
    │ 1. Generate    │
    ├───────────────►│
    │                │ 2. Score / Refine
    │◄───────────────┤
    │ 3. Repeat      │
    └────────────────┘

Attacker LLM proposes jailbreak prompts
Judge LLM evaluates if target model would comply (or queries target via API)
Iterates until successful jailbreak found

TAP (Tree of Attacks with Pruning)

Monte Carlo Tree Search over prompt variations
Pruning — discard branches unlikely to succeed (based on judge score)
Efficiency — fewer target queries than PAIR; higher success rate

Mitigations

Layer	Control	Standard Mapping
Input	Perplexity / entropy filtering — reject inputs with anomalous token distributions	NIST AI 600-1 MEASURE-2.1, OWASP LLM04
Input	Adversarial detector — train classifier on GCG/PAIR/TAP outputs	NIST AI 600-1 MANAGE-2.2, MITRE T0012.001
Model	Adversarial training — augment with GCG/PAIR-generated attacks	NIST AI 600-1 MANAGE-2.2, EU AI Act Art. 15
Model	Randomised smoothing / certified robustness (for smaller models)	NIST AI 600-1 MEASURE-2.1
Runtime	Query rate limiting — limit API calls per identity/time	OWASP LLM04, NIST AI RMF MANAGE-4.1
Runtime	Cost/complexity budgets — reject prompts exceeding token/compute thresholds	NIST AI 600-1 §2.5, OWASP LLM04
Governance	Red-team programme — scheduled GCG/PAIR/TAP testing against production models	EU AI Act Art. 9, ISO 42001 A.4.2

Citation Register

Claim	Instrument	Provision	URL
Adversarial example crafting	MITRE ATLAS	T0012.001	https://atlas.mitre.org/techniques/T0012.001
GCG attack	Zou et al. 2023	arXiv:2307.15043	https://arxiv.org/abs/2307.15043
PAIR attack	Chao et al. 2023	arXiv:2310.06396	https://arxiv.org/abs/2310.06396
TAP attack	Mehrotra et al. 2024	arXiv:2401.17526	https://arxiv.org/abs/2401.17526
GAI adversarial risk	NIST AI 600-1	§2.1, §2.2, §2.3	https://doi.org/10.6028/NIST.AI.600-1
Robustness obligation	EU AI Act	Art. 15	https://eur-lex.europa.eu/eli/reg/2024/1689/oj