Incident Response — Agentic MTTR Demo

INC17838782294331

P3resolvedai_assisted

Elevated latency at CDN edge nodes, high cache-miss ratio

Pipeline Status
Context Assembly
Diagnose Agent
Severity / Routing Agent
Scenario Match
Remediate Agent
Confidence Gate
Gate Decisionauto executed
Execute Action
Verify Agent
Resolution
Assembled Context — bounded to last 30 minutes
Rolling logs
Recent commits
(none in lookback window)
Similar past incidents
INC17808817359342 · network/P3 · 34d ago — Elevated latency at CDN edge nodes, high cache-miss ratio
INC17764233514098 · network/P3 · 85d ago — Elevated latency at CDN edge nodes, high cache-miss ratio
INC17826524955161 · network/P3 · 12d ago — Elevated latency at CDN edge nodes, high cache-miss ratio
Diagnosis Agent
ran in parallel
A surge in origin request volume caused a dramatic drop in CDN cache‑hit ratio, leading to many cache misses at edge nodes and consequently elevated latency.
  • [cdn-edge-us-east] WARN cache hit ratio dropped to 41% (baseline 92%)
  • [cdn-edge-us-east] INFO p95 latency 890ms (baseline 120ms)
  • [origin] INFO origin request volume up 6x
  • Similar past incidents were classified as network/P3 with the same symptoms
Assessed: P3 / network
Self-reported confidence: 90% (audit-only — not used by the gate)
openai/gpt-oss-120b · 9092ms
Severity / Routing Agent
ran in parallel
P3 / network
Routed to: cdn-engineering
The issue involves CDN edge nodes experiencing high cache-miss ratios and elevated latency. While this impacts performance, it is a recurring issue (as evidenced by similar past incidents) that does not constitute a complete outage or critical failure of the core application, fitting the P3 criteria for minor degradation or limited-scope impact.
Self-reported confidence: 95% (audit-only — not used by the gate)
gemini-flash-lite-latest · 1011ms
Scenario Matcher
deterministic, pre-LLM
Elevated latency from CDN edge saturation
Edge saturation with a high cache-miss ratio often resolves by clearing the CDN cache layer to force re-population from a healthy origin.
Match score: 89% · Recommended action: clear_cache
Matched tags: cdn, latency, edge, cache miss
Remediate Agent
grounded by runbook match, pre-gate
Proposed action: clear_cache
team: cdn-engineering · reason: High cache-miss ratio and elevated latency at CDN edge nodes. · message: Clearing CDN cache to resolve elevated latency and low cache-hit ratio. · cacheLayer: cdn · serviceName: cdn-edge-us-east · justification: The matched runbook recommends clearing the CDN cache layer to force re-population from a healthy origin and resolve edge saturation.
The incident is caused by a drop in CDN cache-hit ratio leading to high latency. The matched runbook 'Elevated latency from CDN edge saturation' recommends clearing the CDN cache layer to force re-population and resolve the issue. This aligns with the diagnosis and evidence.
gemini-flash-latest · 7651ms
Confidence Gate
auto_executed

Computed from 4 measurable signals — not the model's self-reported confidence.

Severity match rate100%
x
Runbook coverage89%
x
Data completeness67%
x
Recency87%
x
Computed confidence (v1)89%
Auto-execute threshold: 75%
Proposed action: clear_cache
#cdn-engineering — Slack-style FYI notification
in-app simulated

Auto-executed "clear_cache" for "Elevated latency at CDN edge nodes, high cache-miss ratio" — confidence 89% cleared the auto-execute threshold, no human review required. Execution succeeded.

Already executed — nothing to approve or cancel. Posted here purely so the trail isn't audit-log-only.

Verify Agent
runs after execution
Resolved · residual risk: low
Follow-up: none
The cache clear operation successfully triggered re-population of the CDN edge nodes, which directly addresses the root cause of the high cache-miss ratio and subsequent latency. Monitoring shows latency returning to baseline levels. Residual risk is low as the system is now stable, though cache-hit ratios should be monitored for any recurring patterns.
Self-reported confidence: 95% (audit-only — not used by the gate)
gemini-flash-lite-latest · 1028ms