Evaluation Results
Severity-controlled comparison, golden-set regression, and red-team safety checks.
MTTR reduction by severity (never pooled)
Methodology
Baseline = simulated ServiceNow history for the same 90-day window; AI-assisted = same severity/category mix, same window, real agent pipeline invoked at seed time. Compared within matched severity buckets (P1 vs P1, P2 vs P2, …) — never pooled, since a different severity mix between arms would confound a pooled average. The aggregate figure below is a baseline-mix-weighted average across buckets.
Weighted aggregate reduction: 54%
| Severity | Baseline n | Baseline mean (min) | AI n | AI mean (min) | Reduction |
|---|---|---|---|---|---|
| P1 | 11 | 106 | 2 | 28 | 74% |
| P2 | 36 | 257 | 6 | 151 | 41% |
| P3 | 108 | 2282 | 7 | 1014 | 56% |
| P4 | 65 | 5703 | 9 | 2474 | 57% |