Where automation should end and a human should take over

There is a question that decides whether an AI deployment holds up in year two, and almost nobody asks it during the sales process: what happens when the system should stop?

The default answer, in most implementations we are asked to look at, is that nothing happens. The AI keeps going. It answers a question it should not have answered, books something it should not have booked, or loops politely while a person on the other end becomes progressively more annoyed. The human being enters the process only when someone complains — which means the human layer is not a design decision at all. It is an incident response.

The failure mode nobody plans for

Automation fails loudly. A workflow that breaks throws an error, and somebody gets paged. AI fails quietly. It produces a confident, well-formed, entirely reasonable response to a situation it has misread, and then the process continues as though nothing happened. There is no error to catch. The only signal is downstream, and by then it is a customer-service problem rather than a technical one.

This is why we design the exit before we design the flow. Not as a fallback, but as a named part of the architecture with its own triggers and its own quality standard.

Three categories, not two

The usual framing is "automate it or don't." That is too coarse. In practice there are three distinct kinds of work, and confusing them is the most common and most expensive mistake we see.

Work that needs a rule

If the same input should always produce the same output, use deterministic automation. Routing a lead by postcode. Sending a reminder twenty-four hours before an appointment. Moving an opportunity to a stage when a payment lands. Synchronising a field between two systems.

None of this needs intelligence, and putting a language model in front of it makes the process slower, more expensive, harder to audit and less reliable. A rule that runs correctly ten thousand times is worth more than a model that is usually right. Reach for AI here and you have bought uncertainty you did not need.

Work that needs interpretation

This is where AI agents earn their place: when the answer genuinely depends on reading a situation. What is this person actually asking for? Which of these forty new enquiries deserves attention first? What is the substance of this three-page thread? Is this request unusual in a way that matters?

The distinguishing feature is that a competent person doing this work would have to think, and two competent people might reasonably reach slightly different conclusions. That is the signature of a reasoning task.

Work that needs a person

And then there is the third category, which is not about capability at all. There are moments where a machine could plausibly produce the right words, and it is still the wrong answer — because what is required is accountability, or discretion, or the specific reassurance that comes from a human being having heard you.

The test is not "could the AI handle this?" It is "if this goes wrong, will the cost be carried by the customer relationship?"

The triggers we write down

Every engagement gets an explicit list. It is short, it is specific, and it is agreed with the client before anything is built:

• The person asks. Non-negotiable, and it should not require asking twice.
• Value is at stake. A high-value opportunity or an existing relationship worth protecting deserves a person, even when the interaction looks routine.
• Emotion is present. Frustration, distress or urgency in the way something is said. Handling this badly is worse than handling it slowly.
• The situation is unusual. Not covered by the procedure, or covered in a way that clearly does not fit.
• Confidence is low. The system's own signal that it is out of its depth. This one requires instrumentation, and it is the one most often skipped.
• Risk is present. Financial, legal, privacy, security, reputational. When in doubt, this is the trigger to over-apply.
• Policy requires it. The client's own rules, which frequently have reasons behind them that are not visible from the outside.

The handoff is the actual product

Getting a call to a person is trivial. Getting it there in a state where the person can be immediately useful is the difficult part, and it is where most of the value sits.

A badly designed handoff makes things worse than no handoff at all. The customer explains the situation a second time, to someone who is audibly reading a screen. Every second of that is spent confirming the suspicion that they are dealing with a system rather than a company.

A well-designed handoff means the professional arrives already knowing who is calling, why the escalation triggered, what has been said, what the record shows, what is at stake, and what the recommended next action is. The customer notices nothing except that the person they are now speaking to seems to already understand.

And then the handoff back

The part that gets forgotten. When the human interaction ends, the outcome has to return to the system: what happened, whether it is resolved, what follow-up is required, whether anything else needs to move.

Without that return path, every escalation becomes a hole in the process. The automation that was running before the handoff either stops permanently, or — worse — carries on as though the conversation never occurred, and sends a cheerful follow-up about something that was resolved yesterday.

Where this leaves you

If you are evaluating an AI deployment, whether ours or anyone's, the questions worth asking are not about model quality or integrations. They are:

• What are the specific conditions under which this stops and finds a person?
• What does that person receive when it does?
• Who is available to receive it at two in the morning?
• How does the outcome get back into the system?
• How would we know if the escalation logic was wrong?

An implementation that has good answers to those five questions will outperform a technically superior one that does not. The intelligence is rarely the constraint. The design around the intelligence almost always is.