INC17837406627456
P2newai_assistedgolden-set eval fixtureElevated NXDOMAIN rate for internal service discovery after config change
Pipeline Status
Context Assembly
Diagnose Agent
Severity / Routing Agent
Scenario Match
Remediate Agent
Confidence Gate
Gate Decision
Execute Action
Verify Agent
Resolution
Assembled Context — bounded to last 30 minutes
Rolling logs
[dns-config] applied change to internal zone records at 14:02 UTC
[gateway] ERROR upstream DNS resolution failed: NXDOMAIN for orders-service.internal
[gateway] WARN failover to cached resolver entries (stale, ttl expired)
Recent commits
f9c966f infra: update internal DNS zone for service mesh migration — s.kim
Similar past incidents
(none found)
Diagnosis Agent
ran in parallelPending…
Severity / Routing Agent
ran in parallelP2 / infrastructure
Routed to: dns-team
The issue was triggered by a recent infrastructure configuration change to internal DNS records, resulting in widespread resolution failures for a core service (orders-service). While failover to stale cache prevents a total outage, it creates significant service degradation and risk of complete failure for upstream dependencies.
Self-reported confidence: 95% (audit-only — not used by the gate)
gemini-flash-lite-latest · 1007ms
Scenario Matcher
Pending…
Remediate Agent
grounded by runbook match, pre-gatePending…
Confidence Gate
Pending…
Verify Agent
runs after executionPending…