I’m willing to bet you’ve used a bad AI phone system. Perhaps you called a company with a simple question, and a friendly automated agent answered your call, asked all the right questions, and then asked them again…and again…and again. Your attempts to bypass the agent were rebuffed, or if you did finally manage to escape, you had to start all over again from scratch with the person you reached. Both my wife and I have had that experience recently, and it left us frustrated and even a little bit angry.

So, what went into that company’s decision to implement an AI system, and more importantly, where did things go wrong? The technology behind AI attendants has come a long way in the past few years and can be genuinely useful. However, the question is not whether an AI attendant can answer your phones (it can!), but rather whether your implementation will result in a system that helps your customers or one that drives them up the wall.

I’m seeing a lot more interest from clients in building these systems, and one lesson keeps surfacing. Having an AI assistant answer a series of demonstration calls quickly and efficiently is indeed impressive, but when those calls are carefully curated, that’s easy. The harder part is when the calls are not preselected and when the questions become more complex or unstructured. Dealing with those situations requires a lot of careful work, and that’s rarely demonstrated.

The Voice Is the Commodity

When you sit through a demo, what you are really watching is four pieces working together. There is the telephony layer that receives the call and carries the audio, a speech-to-text layer that turns the caller’s words into text, a language model that works out what the call is about and what to do next, and a text-to-speech layer that answers in a natural voice. Custom solutions are available to handle these steps, as are commercial providers such as Vapi, Retell, and Bland.

When you review the commercially available platforms, once you get past the marketing fluff you’ll quickly be struck by how alike they are. All of them route calls by intent, can record and transcribe calls, and connect to your other business systems. They differ in terms of administration tools, implementation and customization options, and in how they bill for usage, but not in whether the voice itself works. That sameness is revealing — the AI voice is a commodity that keeps getting better and cheaper. However, this is not where these deployments succeed or fail.

The Integration Is the Real Project

If the voice is the easy part, the call still has to connect to something, and that “something” is the real work where things can go wrong. In essence, there are two ways to create these integrations. You can use the platform’s tools to connect to your systems. This is the right starting point for most operations, following the “walk before you run” philosophy. You can also build your own custom integrations, which is very doable and often the right answer when the system needs to work with older infrastructure or with software that offers no clean connection of its own.

Whichever path you take, your phone system becomes a key factor that decides how complex the integration work is. A modern hosted system with direct numbers for your staff makes pointing calls at the AI and transferring them back straightforward. An older on-premises system means lower-level configuration, firewall changes, and often one or more conversations with your telecom provider. None of that appears in a polished demo, and those factors have the biggest impact on your implementation timeline and your cost.

Two Things That Matter More Than the Voice

It may sound counter-intuitive for a system that provides voice, but there are two parts of the integration that deserve more attention than the voice. The first is where the AI’s answers come from. A system that is not tightly constrained will eventually invent a policy or quote a price that does not exist (for instance, Air Canada experienced this in a very high-profile, visible way). I won’t deploy a system that isn’t constrained to approved information and built to admit when it can’t answer and pass the caller along — the kind of unconstrained behavior that turns into a reputational risk leaders need to understand.

The second part is what the call leaves behind. A call that satisfies the caller but doesn’t create a record of it — or creates a wrong or misleading one — hasn’t really been handled all the way through. A good deployment creates the right note, ticket, or follow-up in your own systems automatically, so the work can be tracked, followed up on, or analyzed as necessary. That wiring into your systems of record is unglamorous, but it is a big part of what separates a successful deployment from a convincing demo.

Don’t Build Something That Drives Your Customers Crazy

The systems that drive people crazy are almost always the result of design and implementation choices, not limits of the technology. Systems are designed with rigid menus that will not let someone simply say what they need, agents aren’t designed to recognize that they’re looping back to the same question over and over, the path to reach a human is hidden…all of these are choices, albeit often unintentional ones.

The good news is that you can design each of those problems out before you launch, and you should treat doing so as a core requirement rather than a later refinement. Recognizing the specific ways your system could frustrate a caller, and handling each one deliberately, is some of the most important work in the project. Don’t just focus on the “happy path” that gets followed when everything goes right; pay as much or more attention to handling the unhappy path when things don’t. The goal is not an AI that handles every call, but a system your customers do not resent.

Where It Fits, and Where It Doesn’t

None of this means AI belongs on every line. It fits where calls are high in volume, fairly repetitive, and reasonably bounded. Order status, hours and directions, routing to the right department, and after-hours coverage are natural places to start. However, it is the wrong tool for calls that are low in volume, high in emotion, or open-ended — in those cases, a capable person is both cheaper and better. The strongest systems I have built are narrow on purpose, taking the calls the AI handles well and routing the rest cleanly instead of trying to replace the entire switchboard.

The Takeaway

If there is one thing I’d like you to take away from this, it is to buy the voice and build the integration. AI voice services are a commodity now, and the platforms do that part well. Your real attention belongs on the integration and the design that goes along with it, because that’s what will drive your customers’ experience of calling your business. The impressive demo of a natural-sounding AI answering the phone is the easy part; the plumbing beneath it is the hard work that makes the solution successful.

This wraps up a short run of posts on where to point AI in your own work. Next week I am starting something a little different, a series that looks at putting AI to work industry by industry, beginning on the manufacturing floor where I spent the better part of my career.