Runtime Monitoring

Threats: Model DoS (LLM04), Adversarial Inputs (LLM01), Data Poisoning (LLM03), Supply Chain (LLM05), Excessive Agency (LLM08)
Standards: NIST AI 600-1 §2.5, §2.9; NIST AI RMF MEASURE-3.1/3.3, MANAGE-4.1/4.3; OWASP LLM04; EU AI Act Art. 15/Art. 72/Art. 73; ISO 42001 A.6.1/A.6.3/A.9.1; MITRE TA0008/TA0014

Monitoring Dimensions

Dimension	Metrics	Alert Thresholds
Latency	p50, p95, p99 per endpoint	p99 > 2× baseline for 5min
Throughput	req/sec, tokens/sec	Drop > 30% sustained
Error Rate	4xx/5xx %, refusal rate	> 5% or sudden spike
Token Usage	input/output tokens per request	Outlier > 3σ from session mean
Cost	$/request,$ /session	Budget burn rate exceeded

Behavioural Baselines

Baseline	Collection Window	Update Cadence
Per-user	30 days rolling	Daily
Per-model	7 days rolling	Daily
Per-deployment	14 days rolling	Daily
Global	30 days rolling	Weekly

Features: request rate, token distribution, refusal patterns, tool call sequences, delegation depth, error types.

Anomaly Detection

Method	Use Case	Implementation
Statistical (3σ / IQR)	Latency, token count, error rate	Prometheus + Alertmanager rules
Isolation Forest	Multi-dimensional (tokens + latency + tools)	Nightly batch job; score > 0.7 = alert
LSTM / Transformer	Sequential patterns (tool sequences)	Online inference; threshold on reconstruction error
Change-point (CUSUM / PELT)	Drift in refusal rate, latency distribution	Daily; alert on sustained shift

Adversarial Detection

Signal	Detection	Response
High perplexity	Per-request perplexity > 99th percentile	Flag for review; optional block
Entropy spike	Output token entropy sudden increase	Log + alert; rate-limit session
Refusal pattern change	Session refusal rate > 50% or sudden drop	Session quarantine; human review
Tool call anomaly	Unusual tool sequence / privilege escalation	Block tool call; session termination
Query similarity	Near-duplicate queries from same identity (distillation)	Rate-limit; watermark responses

Drift Monitoring

Type	Metric	Action
Data drift	Input embedding distribution shift (KS test, PSI > 0.2)	Retrain trigger; alert ML team
Concept drift	Accuracy on golden set drops > 5%	Champion/challenger evaluation
Prediction drift	Output distribution shift (entropy, class balance)	Feature store audit; retrain

Incident Response Integration

Severity	SLA	Escalation
Critical (active attack, data exfil)	15 min	Page on-call; auto-block session
High (anomaly sustained > 10min)	1 hour	Slack + email; manual investigation
Medium (drift detected)	24 hours	Ticket to ML team; schedule retrain
Low (baseline update)	Weekly	Automated; no page

Citation Register

Control	Standard	Provision	URL
Monitoring	NIST AI RMF	MEASURE-3.1, MEASURE-3.3, MANAGE-4.1, MANAGE-4.3	https://doi.org/10.6028/NIST.AI.100-1
Robustness monitoring	EU AI Act	Art. 15, Art. 72, Art. 73	https://eur-lex.europa.eu/eli/reg/2024/1689/oj
Performance evaluation	ISO 42001	A.6.1, A.6.3, A.9.1	https://www.iso.org/standard/81230.html
Environmental impact	NIST AI 600-1	§2.5	https://doi.org/10.6028/NIST.AI.600-1
Misuse monitoring	NIST AI 600-1	§2.9	https://doi.org/10.6028/NIST.AI.600-1
Model DoS	OWASP LLM Top 10	LLM04	https://owasp.org/www-project-top-10-for-large-language-model-applications/
Defense evasion	MITRE ATLAS	TA0008	https://atlas.mitre.org/tactics/TA0008
Impact	MITRE ATLAS	TA0014	https://atlas.mitre.org/tactics/TA0014