Incident Response — Agentic MTTR Demo

INC17805975842372

P3resolvedai_assisted

Elevated latency at CDN edge nodes, high cache-miss ratio

Pipeline Status
Context Assembly
Diagnose Agent
Severity / Routing Agent
Scenario Match
Remediate Agent
Confidence Gate
Gate Decisionvetoed
Execute Action
Verify Agent
Resolution
Assembled Context — bounded to last 30 minutes
Rolling logs
Recent commits
(none in lookback window)
Similar past incidents
INC17808817359342 · network/P3 · -4d ago — Elevated latency at CDN edge nodes, high cache-miss ratio
INC17764233514098 · network/P3 · 47d ago — Elevated latency at CDN edge nodes, high cache-miss ratio
INC17826524955161 · network/P3 · -26d ago — Elevated latency at CDN edge nodes, high cache-miss ratio
Diagnosis Agent
ran in parallel
A sudden surge in origin request volume (6× increase) caused cache‑miss rates at the CDN edge to climb to 41% (down from 92%), forcing edge nodes to fetch from the origin and resulting in elevated p95 latency (~890 ms vs 120 ms).
  • [cdn-edge-us-east] WARN cache hit ratio dropped to 41% (baseline 92%)
  • [cdn-edge-us-east] INFO p95 latency 890ms (baseline 120ms)
  • [origin] INFO origin request volume up 6x
Assessed: P2 / network
Self-reported confidence: 90% (audit-only — not used by the gate)
openai/gpt-oss-120b · 1702ms
Severity / Routing Agent
ran in parallel
P3 / network
Routed to: cdn-engineering
The incident involves a significant degradation in CDN cache performance, leading to increased origin load and higher latency. While impact is elevated, the service is still functioning, and similar historical incidents have been categorized as P3. There is no evidence of a total outage.
Self-reported confidence: 95% (audit-only — not used by the gate)
gemini-flash-lite-latest · 985ms
Scenario Matcher
deterministic, pre-LLM
Elevated latency from CDN edge saturation
Edge saturation with a high cache-miss ratio often resolves by clearing the CDN cache layer to force re-population from a healthy origin.
Match score: 83% · Recommended action: clear_cache
Matched tags: cdn, latency, edge
Remediate Agent
grounded by runbook match, pre-gate
Proposed action: clear_cache
team: cdn-engineering · reason: High cache miss ratio causing elevated latency · message: Clearing CDN cache to mitigate high miss ratio and latency · cacheLayer: cdn · serviceName: cdn · justification: Runbook recommends clearing cache; this forces repopulation from healthy origin
Diagnosis shows a surge in origin traffic leading to a 41% cache‑miss rate and high latency. The matched runbook explicitly recommends clearing the CDN cache to address edge saturation, and no contradictory evidence is present. Therefore, clearing the CDN cache is the appropriate remediation action.
openai/gpt-oss-120b · 1127ms
Confidence Gate
vetoed

Computed from 4 measurable signals — not the model's self-reported confidence.

Severity match rate50%
x
Runbook coverage83%
x
Data completeness67%
x
Recency100%
x
Computed confidence (v1)72%
Auto-execute threshold: 75%
Proposed action: clear_cache
Verify Agent
runs after execution
Resolved · residual risk: low
Follow-up: none
The cache clearance successfully addressed the edge saturation and latency issues. While the cache-miss ratio has normalized, the root cause of the surge in origin traffic requires post-incident analysis to ensure it does not recur.
Self-reported confidence: 90% (audit-only — not used by the gate)
gemini-flash-lite-latest · 826ms