AI therapybots (part 3): Chatbot counselors

Part 3 in a series on psychotherapeutic chatbots that include artificial intelligence capabilities. You can read the first two parts here: 

At least a couple startups are exploring AI-based chatbots that offer psychotherapy. Woebot was in the news when AI-superstar Andrew Ng joined the board of directors as chairman. X2AI has built a number of chatbots in different languages, including Tess, which according to The New Yorker can do CBT as well as other types of therapy like motivational interviewing (developed to help problem drinkers overcome their addiction).

AI-based chatbots can potentially overcome some of the drawbacks to in-person psychotherapy:

  • The chatbot can be available around the clock, not just during scheduled sessions
  • A software tool will be much less expensive at the margin for providing therapy — that is, once the investment to build it is undertaken an additional hour of psychotherapy from the chatbot is almost free (except for hosting/processing costs)
  • The chatbot can remember the history of a client’s interactions better than a human can
  • The chatbot can monitor the client for risk of bad outcomes like dropping out of therapy, self-harm including suicide, and lack of improvement — therapists could do this but mostly don’t
  • The chatbot can use various mechanisms to keep the client engaged and potentially reduce dropout/attrition from treatment (though note that in studies of earlier e-therapy programs, dropout was greater than with in-person therapy)
  • The chatbot may be able to keep the client engaged with learning more productive thinking patterns and habits in ways a human therapist cannot. Here’s a chance for the technology community to create habit-forming software for good!

But there are challenges too. As seen with ELIZA, a computerized conversational agent can give someone a feeling of interested empathy, but in general computers aren’t great at conversation. This may improve as AI-based natural language processing improves.

Given that the type of therapy seems not to matter that much, how do we know what chatbots should do to actually promote improvements in mental wellness? Is it just creating a relationship with a therapist that matters, less than the specific recommendations or techniques the therapist does? Does it have to do with the human contact that’s offered? I’m sure there’s plenty that research has to say on this.

I’ve individually reviewed two therapybots so far: Woebot and Wysa. I’ve also reviewed the non-AI non-chatbot mental health app Moodnotes.

All three apps I reviewed use cognitive therapy (CT) techniques at their core: teaching the user to identify automatic thoughts that are bothering them, figuring out the distortions in the thoughts, and then rewriting the thoughts more accurately.

Empirical evidence

There isn’t a lot of research on AI-enabled psychotherapy chatbots. Some research in this area seems to use the term “conversational agent” rather than chatbot.

Right now our culture–and our insurance companies–demand that “evidence-based” approaches be used. Typically this means academic research supporting statistically significant effects, ideally using randomized controlled trials.

Woebot was evaluated in a randomized controlled trial involving 70 adults age 18-28. Participants in the Woebot group received two weeks of Woebot conversational CBT support. The control group participants were an “information-only” control group directed to the National Institute of Mental Health eBook “Depression in College Students.” Those in the Woebot group saw reduced symptoms of depression as measured by the PHQ-9 based on an intent-to-treat analysis (analyzing all participants, not just those who completed the study). The analysis of completers showed that participants in both groups significantly reduced anxiety as measured by the GAD-7. The paper reports results of the intent-to-treat analysis for the PHQ-9 and the GAD-7 for completers only. These were probably where the researchers found significant differences.

This study’s information-only control group is not ideal. This doesn’t tell us if it’s the conversational interface that makes a difference or what the chatbot is talking about (CBT skills).

This is less a criticism of Woebot than of the state of clinical research in mental health today. Much of the research on psychotherapy effectiveness treats the approaches as black boxes, not measuring how people’s skills and knowledge actually changes. Statistics are often curated to support the researchers favored hypothesis. And control groups are either not used or are poorly designed such that they don’t function to establish that a particular psychotherapeutic technique actually works versus establishing that the delivery method is what matters.

This is a huge gap in our understanding of how psychotherapy works (or doesn’t). Is it the contact with a caring human (or chatbot) that helps? Or is it the skills that the person learns and uses? Or an interaction of both?

We need better ways of evaluating mental health interventions, and of understanding exactly how they work, when they do.

How the field is changing

A couple early entrants into this area seem to be in the process refining their feature sets and business models to capture the money that flows via health insurance, moving away from simply providing a conversational AI-based therapybot towards something more broadly supportive of mental health care.

X2AI’s Tess doesn’t seem to be currently available to consumer users. And on its website it calls itself a “customizable EHR.”

Another entrant into the field, Joy, debuted on Facebook Messenger in July of 2016, but seems to have pivoted towards serving as a clinical measurement app that includes passive monitoring of movement, exercise, and sleep.

It’s not surprising to me to see companies that started out in the therapybot space pivoting towards offerings that better address what health payers want and need.

Drawbacks to the chatbot approach

While building an AI-based therapist seems a potentially useful approach to scaling mental health care, the achievements so far in this direction have been limited. Why? Here are two reasons:

  • While AI can be used to help a chatbot infer basic things from human language like what entities are mentioned or what a user wants from a chatbot, it hasn’t progressed to the point where it truly understands language well enough to act as an empathetic listener
  • The CBT approaches that are currently favored for chatbots may be better offered through educational type approaches rather than through conversational interfaces. We could best learn this by research that cracks into the black box of CBT interventions and actually measures how well people use and apply the skills, rather than just checking outcomes like level of depression or anxiety.

In the next post in this series, I’ll consider the future of this field.