Defence in Depth
No single control is sufficient. This page maps a layered defence strategy to the NIST AI RMF Core functions, ensuring each threat vector faces multiple independent barriers.
Primary source: NIST AI 100-1 (AI RMF 1.0), Section 3. NIST AI RMF
Layered Control Matrix
| NIST RMF Function | Input Layer | Architecture Layer | Runtime Layer | Monitoring Layer | Governance Layer |
|---|---|---|---|---|---|
| GOVERN (Accountability, Policy, Culture) |
Acceptable use policy for prompts | Role-based model access; capability tokens | Policy enforcement points (PEPs) | Compliance dashboards; audit logs | Board-level AI risk committee; policy review cadence |
| MAP (Context, Risk ID, Stakeholders) |
Threat modelling per prompt type | Attack surface mapping per deployment | Asset inventory (models, data, tools) | Baseline behaviour per asset | Risk register; stakeholder register |
| MEASURE (Metrics, Testing, TEVV) |
Injection detection rate (TP/FP/FN) | Adversarial test suite (GCG, PAIR, TAP) | Drift detection (PSI, KS, accuracy) | Anomaly detection coverage; MTTR | TEVV programme; red-team cadence |
| MANAGE (Prioritise, Treat, Monitor) |
Block/quarantine on detection | Fail-secure defaults; human-in-the-loop | Auto-remediation (rate-limit, session kill) | Alert triage; escalation paths | Incident response plan; post-incident review |
Threat × Layer Coverage
| Threat | Input | Architecture | Runtime | Monitoring | Governance |
|---|---|---|---|---|---|
| Prompt Injection | Classifier, hierarchy, sanitisation | RAG authorisation, tool scoping | Session quarantine, output encoding | Perplexity/entropy alerts | Red-team injection tests |
| Jailbreak | Multi-model consensus, entropy filter | Constitutional AI, refusal consistency | Session monitoring, refusal tracking | Pattern anomaly alerts | Adversarial training data updates |
| Adversarial (GCG/PAIR/TAP) | Perplexity filter, adversarial detector | Randomised smoothing, ensemble | Query rate limiting, cost budgets | Query pattern clustering | Red-team with GCG/PAIR/TAP |
| Data Poisoning | — | Provenance, signed datasets, robust agg | Influence spot-checks | Validation accuracy drift | Data supply-chain contracts |
| Model Extraction | — | Watermarking, logit truncation | Rate limiting, query anomaly | Distillation pattern detection | ToS prohibition + technical controls |
| Supply Chain | — | SLSA provenance, SBOM, signed artefacts | Admission verification, hash check | Registry anomaly detection | Approved registry; SLSA contracts |
| Agent Hijack | — | Capability tokens, delegation limits | Action confirmation, memory sanitisation | Behavioural baselines | Agent registry, delegation policy |
Defence-in-Depth Principles
- No single point of failure — Each threat faces ≥2 independent controls
- Fail secure — Default-deny on tool calls, output rendering, delegation
- Least privilege — Model, agent, tool run with minimum capabilities
- Observability by design — Correlation IDs across prompt→retrieval→tool→response
- Provenance everywhere — Sign models, datasets, adapters; verify at load/admission
- Human-in-the-loop for irreversible actions — Delete, pay, email, admin require confirmation
- Continuous validation — Red-team monthly; adversarial regression suite on every deploy
Implementation Roadmap
| Phase | Focus | Deliverable |
|---|---|---|
| 1. Foundation | Input validation, schema, length limits; basic logging | Injection detection > 90%; <1% FPR |
| 2. Architecture | RAG authorisation, tool scoping, capability tokens | Zero cross-tenant retrieval; least-privilege tools |
| 3. Runtime | Perplexity/entropy monitoring, rate limits, session quarantine | Anomaly detection coverage > 80% |
| 4. Governance | Risk register, red-team programme, TEVV pipeline | Quarterly red-team; documented TEVV |
| 5. Maturity | Adversarial training, certified defences, automated remediation | Certified robustness claims; auto-remediation |
Citation Register
| Control | Standard | Provision | URL |
|---|---|---|---|
| Govern function | NIST AI 100-1 | GOVERN-1–6 | https://doi.org/10.6028/NIST.AI.100-1 |
| Map function | NIST AI 100-1 | MAP-1–5 | https://doi.org/10.6028/NIST.AI.100-1 |
| Measure function | NIST AI 100-1 | MEASURE-1–4 | https://doi.org/10.6028/NIST.AI.100-1 |
| Manage function | NIST AI 100-1 | MANAGE-1–4 | https://doi.org/10.6028/NIST.AI.100-1 |
| GAI risk areas | NIST AI 600-1 | §2.1–§2.12 | https://doi.org/10.6028/NIST.AI.600-1 |
| Robustness | EU AI Act | Art. 15 | https://eur-lex.europa.eu/eli/reg/2024/1689/oj |
| TEVV | ISO 42001 | A.7.1–A.7.3 | https://www.iso.org/standard/81230.html |
| Incident response | ISO 42001 | A.6.2 | https://www.iso.org/standard/81230.html |
| Supply chain | SLSA | v1.0 | https://slsa.dev/ |
| Adversarial tactics | MITRE ATLAS | TA0002, TA0004, TA0005, TA0008, TA0012, TA0013, TA0014 | https://atlas.mitre.org/ |