Data Poisoning

MITRE ATLAS: T0012.002 — Poison Training Data
OWASP LLM Top 10: LLM03 — Training Data Poisoning
NIST AI 600-1: §2.3 (Dangerous Content), §2.4 (Data Privacy)
EU AI Act: Art. 10 (Data governance), Art. 15 (Robustness)

Taxonomy

Type	Goal	Mechanism	Detectability
Availability / Indiscriminate	Degrade overall model performance	Inject mislabelled / noisy samples across classes	Moderate — elevated loss on clean validation
Targeted / Clean-label	Cause specific misclassification on chosen inputs	Craft poison samples that look correct but shift decision boundary	Hard — labels appear correct
Backdoor / Trigger-based	Cause misclassification only when trigger present	Embed trigger pattern (pixel patch, token sequence) in poison samples; model learns trigger→target mapping	Hard — normal accuracy preserved
Fine-tuning / RLHF Poisoning	Align model to attacker preferences	Poison preference data or reward model training set	Hard — subtle distribution shift

Primary source: MITRE ATLAS T0012.002. ATLAS

Threat Vectors

Public dataset contamination — attacker uploads poisoned samples to Hugging Face Datasets, Common Crawl, LAION
Internal data pipeline compromise — insider or CI/CD breach injects poison into proprietary training corpus
Synthetic data poisoning — attacker controls LLM generating synthetic training data (e.g., for instruction tuning)
RLHF preference poisoning — attacker manipulates human feedback or reward model training data

Detection Methods

Method	Principle	Limitation
Influence functions	Approximate LOO influence of each training point on test loss	Computationally expensive; approximate
Spectral signatures	Poisoned clusters exhibit distinct covariance structure in representation space	Requires clean reference; false positives on rare classes
Activation clustering	Poison activations cluster separately from clean	Assumes poison forms distinct cluster
k-NN / neighbourhood	Poison samples have anomalous neighbours in feature space	High-dim curse; choice of k
Robust statistics (trimmed loss)	Trim top-α% loss samples per epoch	May discard hard clean samples
Data provenance / hashing	Cryptographic hash + signed attestation per data batch	Requires upstream adoption; not retroactive

Mitigations

Layer	Control	Standard Mapping
Ingest	Data provenance — signed hashes, SBOM for datasets	NIST AI 600-1 GOVERN-4, ISO 42001 A.9.1
Ingest	Allow-list approved data sources; block unknown scrapers	NIST AI RMF MAP-2.3, ISO 42001 A.8.2
Training	Robust aggregation (Krum, Median, Trimmed Mean) for federated / multi-party	NIST AI 600-1 MANAGE-2.2
Training	Differential privacy (DP-SGD) — bounds influence of any single sample	NIST AI 600-1 MANAGE-2.2, §2.4
Validation	Clean held-out validation set (never touched by training pipeline)	ISO 42001 A.7.2, EU AI Act Art. 10
Monitoring	Influence-function spot checks on high-stakes predictions	NIST AI RMF MEASURE-3.1
Governance	Data supply-chain contracts — liability for poisoned data	EU AI Act Art. 25, ISO 42001 A.8.3

Citation Register

Claim	Instrument	Provision	URL
Poisoning taxonomy	MITRE ATLAS	T0012.002	https://atlas.mitre.org/techniques/T0012.002
Training data poisoning	OWASP LLM Top 10	LLM03	https://owasp.org/www-project-top-10-for-large-language-model-applications/
GAI data risk	NIST AI 600-1	§2.3, §2.4	https://doi.org/10.6028/NIST.AI.600-1
Data governance	EU AI Act	Art. 10	https://eur-lex.europa.eu/eli/reg/2024/1689/oj
Robustness	EU AI Act	Art. 15	https://eur-lex.europa.eu/eli/reg/2024/1689/oj