Four ICF assessors sat down with a stack of coaching transcripts and marked them the way they mark everybody else: against the competency framework, using the behaviourally anchored rating scales, as if a professional human coach were on the other side of the conversation.

A professional human coach was not.

The coach was Alpina, a commercially available AI coaching agent developed by the company evoach, text-based and running on OpenAI's GPT-4 Turbo with additional prompt coding by the developers. Forty-three managers volunteered to be coached by it. They were middle managers leading small teams, drawn from three organisations in technology and engineering, each invited to bring a real workplace challenge to the conversation.

Clients chose whether to save or delete their transcript. Saved transcripts were randomly assigned to one of four ICF assessors with the marking guide, and assessors were told to mark them as if they were human coach submissions. Between them, those assessors had marked more than 500 ICF scripts.

The study, by Jonathan Passmore, David Tee and Rebecca Rutschmann, ran in Coaching: An International Journal of Theory, Research and Practice.

The parts it passed

The headline result is the one the industry has been bracing for. The results indicated the AI coaching agent was able to demonstrate many elements of both ICF ACC level and PCC level coach competencies. On evoking awareness it achieved clear ACC and was judged to be bordering on PCC.

On ethics, the machine's advantage was structural. There was no contra-evidence that the AI coaching agent broke confidentiality; as a non-human it had no conflicts.

Then came the line that should genuinely sting. Assessing coaching mindset, one assessor wrote that the AI was "excellent at maintaining a non-judgemental and open-minded stance," consistent and positive, fostering an environment of continuous improvement and a growth mindset approach.

That is the pitch on half the coaching websites in America. A bot cleared it.

The one thing it could not do

Competency 5 is Maintains Presence. One of its markers is plain: the coach allows for silence, pause or reflection.

Here the assessors stopped being impressed. While the AI agent showed aspects of maintaining presence, it had several shortcomings — and its biggest shortcoming was its inability to allow for silence, pauses and reflection.

The bot was instead always keen to either summarise or move the conversation forward, without pause or space for reflection. On coaching mindset it reached ACC standard, with the main feedback being to reduce the amount of summarising and give the client more space.

The other gaps cluster around the same hole. It could not create its own metaphors to help a client see their situation fresh, and assessors judged its responses to "often sound like textbook answers." What looked like empathy was felt by assessors to be "often formulaic," possibly because the agent played back the client's words but brought no personal experience into the room. And one assessor said the bot lacked the nuanced judgement of an experienced human coach, with concern about whether it could recognise and respond to subtle ethical issues such as managing a personal relationship at work.

Strip the academic register off and the finding is almost rude in its simplicity. The AI talks too much.

Then they asked the clients

A second study from the same research group went at the question from the other end: not how good is the AI, but how does it feel to be coached by one.

A small-scale quasi-experimental pilot with 63 white-collar workers used a custom questionnaire to compare clients engaging with a human coach or with Alpina, across developing new insights, working alliance, goal attainment, commitment, trust, confidentiality and shame. Twenty-eight participants, mean age 41, went to the AI; 35, mean age 47, went to a human. The 28 human coaches were ICF- or EMCC-accredited professionals.

After a single session, clients reported higher scores with the human coach than with the AI coaching agent on all factors. That paper, in the Journal of Work-Applied Management, was authored by Passmore, Tee, Gina Palermo and Rutschmann.

Two caveats, loudly. Both samples are small. The authors caution that the sample was limited and the design exploratory. And the technology graded here is already a generation old.

The uncomfortable part

Read the two studies together and the line falls in a strange place.

AI is already competent at the things coaches put on their sales pages: non-judgement, structure, consistency, confidentiality, good questions, an open-minded stance at 11pm on a Tuesday. It is not yet competent at the thing almost nobody sells, because it is impossible to photograph.

Space.

Which raises a question worth sitting with before your next session. When a client goes quiet and starts writing in their notebook, what do you do? The paper cites the human move: noticing the client is making notes and telling them to continue when they're ready.

Most coaches don't do that. Most coaches summarise. Most coaches reframe. Most coaches ask the next clever question, because the next clever question is what they were trained to be good at and what clients visibly reward.

That is precisely the behaviour four ICF assessors marked the robot down for.

The defensible skill in 2026 is not asking better questions than a model trained on everything. It is being able to say nothing for eleven seconds while someone finds their own sentence — and not needing the credit.