KPIs and kill switches

• Gate 1 (before cutover): Does the shadow-run accuracy clear the error-cost threshold you set for this specific process in The pilot-build sequence? If not, extend the shadow-run or narrow the automation's scope — don't cut over on hope.
• Gate 2 (after the first month): Does the logged number — hours saved × your real fully-loaded hourly cost, minus tool and API cost — show a payback period inside the ceiling you set going in? If it doesn't, the process either wasn't a good candidate or was mis-scoped; revisit Auditing your own operations rather than assuming the next automation will do better with the same approach.
• Gate 3 (ongoing): Is the human-escalation path actually being used at a reasonable rate — not near-zero, which suggests nobody's really monitoring it, and not near-100%, which suggests the automation isn't actually doing the work? Track this explicitly rather than assuming silence means success.
• Gate 4 (ongoing, for anything customer-facing): Define your own "resolved" the way What actually automates shows Intercom and Zendesk define theirs — a closed ticket where the customer gave up isn't a win. Measure against your own definition, not a vendor dashboard's.
• Kill/pivot signal: if an automation's escalation or override rate stays high past its shadow-run window instead of settling down, kill it — you're now paying for the tool and still doing the labor, which is worse than not automating at all.

#Why this discipline isn't optional

Gartner, polling more than 3,400 organizations actively investing in agentic AI, predicts more than 40% of agentic AI projects will be canceled by the end of 2027 — with the named reasons being deployment without clear strategy and inadequate governance for what happens when something goes wrong, not a shortfall in the underlying technology. [Established — Gartner press release, June 2025, named methodology] The gates above exist specifically to keep a small, self-built automation program out of that failure pattern: cheap to build usually also means cheap to skip monitoring on, and skipped monitoring is exactly the governance gap Gartner's prediction is describing at enterprise scale. Failure modes covers what that looks like concretely when it goes wrong in an operator's own business rather than a large enterprise's.