For four years, the best sentence in every AI coaching pitch deck came from a paper with Nicky Terblanche's name on it.

PLOS ONE, 2022. Two equivalent longitudinal randomised controlled trials measuring goal attainment over ten months, one using human coaches, one a replication using an AI chatbot coach. Both beat their control groups. And at the end of the trials, the AI coach was as effective as the human coaches.

The authors then wrote the three implications that launched a thousand landing pages: AI coaching could be scaled to democratise coaching, AI coaching could grow demand for human coaching, and AI could replace human coaches who use simplistic, model-based coaching approaches.

That third one has been read aloud at conferences ever since, usually by someone selling software.

Terblanche is now a co-author on the study that complicates it.

The first real head-to-head

De Haan, Terblanche and Nowack published in Human Resource Development International, volume 29, issue 4, pages 754 to 783. It went online on 28 February 2026.

The design is the part that matters. With N = 114 coachees, accredited human coaches were compared directly with automated AI coaches for the first time, with outcomes measured using established and validated progress psychometrics covering goals, motivation, resilience and wellbeing. The trade publication People Managing People reports the 114 coachees sat inside a single global organisation.

Three legs: human coaching, chatbot coaching, and a waitlist control. The human coaching and waitlist groups had agreed sessions of one hour weekly; the chatbot group was asked to consult the coaching chatbot once a week for an hour. And unlike the 2022 work, the subjects were senior managers rather than students.

The finding, in the authors' own abstract: the study demonstrated substantial effectiveness in diverse coaching outcomes but only for human coaching, with effect sizes in the mid-to-high range, and strong predictions of success by the coachee's initial self-efficacy and hope.

De Haan's own summary of the trial is blunter. Human coaching produced significant, medium-to-large improvements across nearly every outcome measured, including goal success, self-efficacy, perceived stress and the coaching relationship itself. AI-chatbot coaching did not significantly outperform the control group on almost any outcome, including goal attainment, the one area earlier chatbot research had shown promise in.

People Managing People's read on the AI arm: it "didn't beat the control group, the people who got no coaching at all" on a single primary measure.

What the paper says about its own contradiction

The authors do not hide the collision. The paper states that its findings do not support the earlier 2022 study, which found AI helped significantly with goal achievement only, using an identical measure of goal success. This time the only significant difference for the AI group was on General Self-Efficacy, precisely what the 2022 study did not find. The paper then reports that self-efficacy difference disappeared.

They also tested the obvious objection, that the bot group simply used it less. Regressing the AI results on time spent with the chatbot improved things only slightly, with the Control–AI differences remaining under the .05 threshold.

One explanation the authors float is a nocebo or outcome-expectation effect: participants in the AI group knew others were receiving something else.

There is also an engagement number that should make every AI coaching buyer pause. Attrition was roughly ten times higher in the AI-coaching group than in human coaching, even with regular reminders.

The 2022 study wasn't wrong. It was asked something smaller.

This is the framing coaches should actually use, because it survives scrutiny better than "AI doesn't work."

The 2022 study handed the AI a narrow, structured job: move someone through a defined goal-attainment protocol. The 2026 study asked it to be the coach.

Look at the earlier chatbot trial on its own terms. A replication RCT tested a chatbot coach called Vici, with an experimental group of 75 using it for six months against a control group of 94. The experimental group showed a statistically significant increase in goal attainment, while all other measures came back non-significant.

One outcome moved. The rest didn't. That was always the shape of the evidence, even in the study that got quoted as a victory. And the 2022 authors said as much at the time, cautioning that "AI's lack of empathy and emotional intelligence make human coaches irreplicable" — a line that somehow never made the sales deck.

The finding that should bother coaches too

The second result is less comfortable, and it cuts at the whole industry, not just the software.

Success was predicted most strongly by the coachee's starting self-efficacy and hope. The people who got the most from coaching walked in already believing they could grow. The employees who most need development support, the ones without that foundation, are exactly the ones an unsupervised bot serves worst.

The authors' explanation is a co-regulation model, which predicts better results for human coaches. Two nervous systems in a room, or at least on a call. A machine can hold the protocol. It apparently cannot hold the person.

The commercial implication is uglier than the research one. If corporate buyers read this properly, the cheap AI tier goes to the employees with the least confidence and the human coach goes to the high-potential list, which is the opposite of what the evidence supports. As People Managing People puts it, deciding who gets a person and who gets a chatbot is not a procurement question.

What to do with it on Monday

Send the citation, not the summary. The paper opens by noting executive coaching is estimated at a $4.56 billion global industry, which is roughly the size of the pot AI coaching vendors are pitching against.

When a prospect says they'll just use ChatGPT, the honest answer is now specific: for a narrow, well-defined goal with a motivated, confident person, a structured bot has produced measurable movement. For open-ended development inside a real organisation, the first direct comparison found nothing above no coaching at all.

One study is one study. This one has 114 people, three arms, a waitlist control and a co-author arguing against his own most-quoted result.