Why Building Your Own AI Coach Is the Easy Part

Over the past several years, I've built coaching agents across newer agent platforms like Vapi or Eleven Labs, enterprise systems like Microsoft Foundry, and frameworks like LangGraph.

In late 2026, the architecture is starting to converge.

We now have fairly standard ways to give an agent tools, memory, and state. Evaluation is becoming part of the normal development process. Model Context Protocol (MCP) and similar standards are making it easier to connect agents to the systems around them.

All of this means we're getting very good at assembling something that looks like a finished product, and the harder question becomes whether it's actually good.

The components are getting easier; the system is still custom

Modern agent platforms are genuinely impressive.

Microsoft Foundry, for example, can provide much of the infrastructure needed to build and operate an enterprise agent. LiveKit does something similar for realtime voice.

This is real progress. The basic components are increasingly available off the shelf.

But I think agents are beginning to look a little like websites.

Website builders made publishing a website easy. They didn't make every website good. Decades later, companies still pay specialists to design and build important digital experiences because the hard part was never simply getting a page online.

Agents seem to be heading in the same direction.

You can assemble the components quickly. The slower work is teaching the system exactly how you want it to behave, then making sure it keeps behaving that way.

For coaching, that distinction matters.

A coach shouldn't simply give a factually correct answer. It may need to resist giving an answer at all.

A chatbot that sounds like a coach isn't necessarily a coaching system

A good prompt can make a model sound remarkably like a coach.

It can ask thoughtful questions. It can reflect what someone has said. In a short demo, it can be difficult to tell the difference between that and a more purpose-built system.

The difference tends to appear over time.

Imagine an executive repeatedly asking the agent, “Just tell me what you think I should do.”

A prompted chatbot may observe a rule to 'avoid giving advice' until pressured by the executive. But a coaching system can understand where the conversation is, recognize that the boundary is being tested repeatedly, and hold the line because of a defined ruleset at the conversation node.

It can also be tested against this situation before a real person finds it.

Simply put, it's a different product.

The architecture underneath might include state, memory, and a conversation graph. But those aren't really the point. The point is that the system has been designed around a standard of behavior, rather than simply prompted to sound right one message at a time.

Most of the learning happens after the demo

This is the part I think organizations can underestimate.

If you want to test whether employees will use an AI coach, I would build a prototype quickly. The tools are good enough that there is little reason not to.

The mistake is assuming that the prototype has answered the harder questions.

Real users will find situations you didn't test.

One conversation will expose a failure. You fix it. Ideally, that conversation then becomes part of the evaluation suite so the same problem doesn't return when you change the model six months later.

Over time, those examples accumulate.

The system gets better not only because the models improve, but because you learn more about what can go wrong and turn that learning into something testable.

That is difficult to see from the outside.

Two coaching agents can look almost identical in a ten-minute demo but run on very different systems underneath. In production, at scale, there's a real difference.

Coaching adds another layer: what does “good” mean?

General AI products already have useful ways to test whether an answer is accurate or safe.

Coaching has another problem.

Was it good coaching?

You have to answer that question before you can evaluate it.

This is one reason I've found the work happening at CoachBot interesting.

Nicky Terblanche and Andrew Brown's research into directive versus non-directive AI coaching (Frontiers in Psychology, 2026) is a good example. Whether an AI coach should become more directive at a particular moment isn't something I would want hidden inside someone's prompt because it seemed reasonable at the time. It's a coaching question that deserves evidence.

That research feeds into a broader quality process at CoachBot chaired by Chief Coaching Officer Jonathan Reitz and its Responsible AI Coach Quality Council.

I think that matters.

A serious coaching platform should have a point of view about what good coaching looks like. It should explain that point of view and test its system against it.

Otherwise, you're still relying heavily on a general-purpose model to do a convincing impression of a coach.

The technology isn't really the moat

Most of the underlying technology can now be built or bought.

That's a good thing.

I don't think CoachBot, or any other serious AI coaching company, wins because it has discovered a secret way to store memory or call a language model.

The advantage is more likely to come from what's built on top of those components.

It's the coaching methodology. It's the evaluation system around it. And, over time, it's the collection of real situations that have taught the system where coaching becomes difficult.

That work also has to continue.

A model changes, and behavior moves slightly. A new feature creates an unexpected interaction. A customer introduces a use case the system hasn't seen before.

Someone has to notice, decide what good should look like, and make sure the next version does it better.

That is normal product work. But it is also work that an organization takes on when it decides to build the whole capability itself.

I don't think this is really build versus buy

The more mature this market becomes, the less useful that distinction feels to me.

We don't usually ask whether a company should “build or buy a website” anymore.

They might use a commercial platform and an internal design team. They might hire a specialist for the difficult parts. What matters is choosing what they want to own and where existing expertise gives them leverage.

I think AI coaching will work the same way.

If coaching is an experiment, build something this week.

If it is going to become a meaningful capability inside the organization, I would ask a different question: where do you actually want to develop expertise?

You may want to own your employee experience, your data, and the way coaching connects into your organization.

I'm less convinced that most companies need to spend years independently developing a coaching methodology, a quality framework, and a growing library of edge cases if a specialist has already done much of that work.

That's where I think platforms like CoachBot become interesting.

Not because building an AI coach yourself is impossible. Quite the opposite: building one is getting easier all the time. Knowing what good coaching looks like, proving that your system delivers it, and keeping it there is the harder part.

Estimated read time: 6 minutes

David Meehan

Al product strategist and advisor to CoachBot AI