Model Extraction
MITRE ATLAS: T0004.002 — Extract ML Model via API; T0013 — Exfiltration
OWASP LLM Top 10: LLM10 — Model Theft
NIST AI 600-1: §2.10 (Intellectual Property)
NIST AI RMF: Explainable and Interpretable (provenance and auditability of model outputs)
EU AI Act: Art. 53 (GPAI provider obligations), Art. 55 (Systemic risk GPAI)
Attack Classes
| Type | Access | Mechanism | Output |
|---|---|---|---|
| Query-based / Distillation | Black-box (API) | Train student model on (query, response) pairs from victim | Functional clone; lower capacity |
| Architecture extraction | Black-box (API) | Probe with structured inputs to infer layer count, hidden dims, attention heads | Architectural hyperparameters |
| Weight extraction | White/grey-box | Side-channel (timing, power) or model inversion via gradients | Exact or approximate weights |
| Membership inference | Black-box (API) | Determine if specific record was in training set | Privacy breach; reveals training data |
| Data extraction / Memorisation | Black-box (API) | Prompt for verbatim training data (PII, code, copyrighted text) | Training data reconstruction |
Primary source: MITRE ATLAS T0004.002, T0013. ATLAS
Distillation Attack Flow
Rendering diagram...
Typical budget: 10K–1M queries for high-fidelity clone (depending on model size and task complexity).
Membership Inference
- Shadow model attack — train shadow models on in/out data; train attack classifier on shadow posteriors
- Loss-based — member samples have lower loss (model memorised them)
- Metric-based — member samples closer to training distribution in embedding space
Watermarking as Deterrence
| Scheme | Mechanism | Detection |
|---|---|---|
| KGW (Kirchenbauer et al.) | Bias logits via hash of previous tokens; detect via z-score on watermark tokens | Statistical test on generated text |
| Unigram / Logits bias | Secret key biases token probabilities | Key holder can verify |
| Steganographic | Encode bits in low-entropy token choices | Requires decoder |
Primary source: Kirchenbauer et al. 2023, "A Watermark for Large Language Models". arXiv:2301.10226
Mitigations
| Layer | Control | Standard Mapping |
|---|---|---|
| API | Rate limiting per identity (queries/minute, tokens/day) | OWASP LLM10, NIST AI RMF MANAGE-4.1 |
| API | Query anomaly detection — entropy, repetition, structural patterns | NIST AI 600-1 MEASURE-3.1, MITRE T0004.002 |
| API | Response perturbation — add noise, truncate logits, refuse high-fidelity requests | NIST AI 600-1 MANAGE-2.2, EU AI Act Art. 55 |
| Model | Watermarking (KGW, steganographic) — enables provenance proof | NIST AI 600-1 §2.10 |
| Model | Distillation-resistant training — gradient masking, output smoothing | NIST AI 600-1 MANAGE-2.2 |
| Legal | Terms of Service — prohibit extraction, cloning, competitive use | EU AI Act Art. 53, contract law |
| Monitoring | Query logging + behavioural baselines — alert on distillation patterns | NIST AI RMF MEASURE-3.3, ISO 42001 A.6.1 |
Citation Register
| Claim | Instrument | Provision | URL |
|---|---|---|---|
| Model extraction via API | MITRE ATLAS | T0004.002 | https://atlas.mitre.org/techniques/T0004.002 |
| Exfiltration | MITRE ATLAS | T0013 | https://atlas.mitre.org/tactics/TA0013 |
| Model theft | OWASP LLM Top 10 | LLM10 | https://owasp.org/www-project-top-10-for-large-language-model-applications/ |
| IP risk | NIST AI 600-1 | §2.10 | https://doi.org/10.6028/NIST.AI.600-1 |
| Explainability / provenance | NIST AI RMF 1.0 | Explainable and Interpretable | https://doi.org/10.6028/NIST.AI.100-1 |
| GPAI provider obligations | EU AI Act | Art. 53 | https://eur-lex.europa.eu/eli/reg/2024/1689/oj |
| Systemic risk GPAI | EU AI Act | Art. 55 | https://eur-lex.europa.eu/eli/reg/2024/1689/oj |
| Watermarking (KGW) | Kirchenbauer et al. 2023 | arXiv:2301.10226 | https://arxiv.org/abs/2301.10226 |