App Review: Woebot

Woebot, a chatbot to promote mental health, received tons of buzz when data science superstar Andrew Ng announced he was joining its board of directors, in an article titled, “Woebot: AI for mental health.” But Woebot doesn’t, at this point, include much obvious AI. And it’s unclear that adding AI will improve it.

Is the entire therapybot effort misguided? Woebot makes me wonder if it is. Given the lack of any current AI capabilities that actually understand natural language (versus simply parsing it), a bot interface seems the wrong way to build digital mental health support.

Woebot uses a chatbot interface to guide the user in learning and using cognitive behavioral therapy skills with the goal of improving mental health. It’s billed as a “mood management program.”

Ng says “mental health may be the killer app for chatbots.” I disagree. Natural language processing technology hasn’t progressed to the point where this is a useful interaction paradigm for promoting mental wellness. I suspect that digital technology will allow us to explore radically new ways of promoting mental health that don’t involve simulating a therapist.

Related: review of evidence for online psychotherapy offerings

Woebot: What works

  • Guidance on identifying cognitive distortions
  • Gratitude tool – although I can’t seem to find my way back to it!

Woebot: What doesn’t work

  • Preteen-style communications
  • Mood tracking via daily emoji selection
  • Basic conversational capability
  • Lack of structured way to navigate capabilities

Identifying cognitive distortions

The meat of Woebot is not the chatbot capability or the mood tracking. It’s the cognitive behavioral therapy (CBT) and other mental wellness coaching it provides. CBT teaches behavioral and cognitive skills that are intended to help the user think and behave and thereby feel better. CBT and other therapist-led approaches have been found to be effective in numerous studies.

Woebot includes guidance in spotting cognitive distortions, rewriting thoughts, expressing gratitude, goal setting, and practicing mindfulness, among other tools. I’m not sure what else it does, as there’s no menu that shows you possibilities. You just get offered various tools as you interact with Woebot. This could be a source of serendipitous surprise but it also felt a little like I was wandering blind without knowing what was available.  It also made it hard to do what I wanted with the app rather than letting Woebot itself decide what I would do when I launched it each day.

Identifying cognitive distortions in automatic thoughts and then revising thoughts without the distortions is key to the effectiveness of CBT. Woebot calls this its Anxiety Buster tool. This is a very useful capability, but it’s too bad it’s buried underneath the annoying chatbot interface and ineffective mood tracking (see below).

Woebot asks you to specify three thoughts that are creating anxiety. Then it guides you through possible cognitive distortions one by one, until you’re ready to rewrite the thought without distortions.

Distortions that Woebot helps you identify include:

  • Emotional reasoning – using feelings as guidelines for truth
  • Mind-reading – assuming you know what others are thinking
  • Fortune telling – predicting things will turn out badly (sometimes known as catastrophizing)
  • Should statements – punishing yourself with “shoulds,” “ought-to”s, or “supposed tos”
  • All-or-nothing thinking – thinking in black or white categories
  • Overgeneralization – arriving at a general conclusion based on a single piece of evidence

There are probably a handful of others too–each time I use the tool it appears to use a different script that takes me through a variety of possibilities.

As you run through possible cognitive distortions for each thought, I imagine Woebot’s AI capabilities are keeping track of which automatic negative thoughts are associated with which cognitive distortions so that eventually it can suggest distortions itself based on the thought specified by the user. I can imagine a machine learning model that handles this.

I find it hard sometimes to pick out the cognitive distortions that apply to a particular thought. I’ve had the experience in learning CBT skills before that many of the cognitive distortions feel like slight variations on one another: filtering out the positive, overgeneralizing from negative evidence, all-or-nothing thinking, … what exactly is the difference across these? Any machine learning model that learns off of this training data is likely not going to do a great job, at least with the overlapping categories.

Gratitude tool has disappeared

I liked the gratitude tool the couple times it was offered to me by Woebot, but due to a buggy system, I wasn’t able to return to it. I launch Woebot, fend off its chatty advances, and ask for my “Toolbox” where I’m supposed to get access to methods I’ve already learned.

I choose Methods and then Gratitude Journal. From there things get strange, it asks me, “Which Flashpoint Tool would you like to use?” The two options are Five and Ice. I choose Ice, still hoping to get to the Gratitude Journal. From there I enter some strange parallel universe involving ice. “Do you have ice?” Woebot asks. Clearly this is not the gratitude journal!

2018-06-01 17.08.58.png

“It’s like chatting with a pre-teen girl”

Woebot presents itself as a chatbot, but it is really more of a scripted interaction that looks like iMessages, designed in a way that discourages free text conversation Most of the time Woebot gives you one or a few options to respond with rather than encouraging and responding well to free text responses to its questions.

One reviewer on the app store said that interacting with Woebot is like “texting with a preteen girl.” I’m not sure about pre-teens, but I text with my two teenage daughters regularly, and their humor is witty and sardonic, not silly and distracting like Woebot’s. My girls do share animated gifs and videos pretty regularly, but they do it ironically. Woebot doesn’t do irony.

hedgehog

Granted, the hedgehog is very cute! But I find this kind of interaction more distracting than helpful, especially if I’m feeling overwhelmed by strong negative emotions.

Apps aimed at the millennial crowd seem to take on this casual communications style, for better or worse. I wrote data science content recently for the Smartly online learning platform, and the use of casual preteen-style humor and funny videos was required in all our lessons. I’m middle-aged so maybe I just don’t get it! But I find this sort of communication distracts me from the meat of the content.

Mood tracking

With notifications turned on, Woebot pings you to check in once a day. During these check ins, Woebot asks you what you’re doing, and prompts you for an assessment of your mood, using emojis:

IMG_1285

After a few check-ins, Woebot presents a not-terribly informative graph of mood for the past week:

IMG_1344

A couple things are problematic about this approach.

First, emojis aren’t good representations of emotion. Lisa Feldman Barrett proposes that the more we get emotionally granular, the more wellness we will have. But emojis aren’t specific to emotions. What do these three emojis represent, for example? How do they differ in valence (positive or negative affect)? Or in arousal (level of energy)? What about in conceptual content (Barrett says that emotions have cultural and conceptual meaning)?

emoji-happy

The official descriptions of these two emoji are “grinning face with normal eyes” and “smiling face with squinting eyes.” Maybe this is again a reflection of my inability to communicate in modern millennial language, but I don’t really get a difference in mood between the two. I like the one with red cheeks better so I usually choose it if I’m feeling mostly positive. I also like the thinking emoji, but again, I’m not certain if that’s a more or less positive feeling than the grinning and smiling emoji above.

Second, people’s moods fluctuate during the course of the day, often with peaks in the morning and evening, and a dip in the afternoon. This is well-covered by Daniel Pink in his latest book When. Here’s a relevant figure from that book:

circadian-mood

I didn’t check in at the same time every day (even if Woebot notified me to check in at the same time). So my mood could reflect within-day variation rather than a difference in mood. Or it could reflect some momentary happiness or stress.

Psychologists have spent all sorts of effort trying to come up with good ways of measuring subjective well-being, happiness, and mood (see, for example, experience sampling, ecological momentary assessment, and the day reconstruction method). Woebot’s mood tracking doesn’t reflect the complexity of measuring people’s moods. I’ll cover that topic in more detail in a later post.

Woebot says, “it’s very helpful to reflect on your moods each week” but I found this implementation of mood tracking and reflection not very helpful at all. I like Moodnotes’ approach much better, though it as well suffers from a variety of weaknesses.

Why a chatbot?

I question whether a chatbot interface is appropriate for this material. Typically, psychotherapy has been offered one-on-one via another person, but that doesn’t mean that’s the best way to provide it, or that we should try to reproduce it in digital form. Using the fiction that you’re interacting with another person limits the utility and the possibilities of electronically-based mental health guidance.

This reminds me of how we used to spend lots of time organizing emails into folders, just like we filed paper documents in the real world. Then Gmail came along and blew that idea out of the water with the archive-and-search paradigm. Digital technology can do things better than real-world technology, but only if you leverage its unique strengths.

Computer programs aren’t currently very good at communicating in natural language, except in very constrained domains. I wonder if another kind of interface would work better.

A further problem with this interface is it can be hard to get to the capability you want to get to. I wanted to try out the Anxiety Buster cognitive distortion identification again (for review purposes!), but instead I ended up nearly triggering Woebot’s crisis response system:

IMG_1349

Unlike non-chatbot style interfaces, it’s not always easy to get where you want to go with Woebot, and you sometimes get places you don’t want to go. Furthermore, without menus it’s hard to know the range of capabilities and guidance that Woebot offers.

No AI here

Woebot shows very little ability to understand natural language. Even when it’s engaging in restricted interactions it’s incapable of understanding basic communications.

Here was one particularly nonsensical interaction:

IMG_1302

I’m not sure why Woebot asked me about costumes, and then ignored when I offered that it could dress up like a zebra. This kind of thing feels like a waste of my time. I suppose the idea is to create some sort of chatbot that I feel connected to, but I found the pretense that there was someone else there offputting, not helpful to improving my sense of wellness.

Data privacy and security

Data privacy is critical to mental health apps. Woebot says data is not linked to you in any identifiable way. “It is de-identified and anonymous.” They say that conversations with Woebot “are not shared with any other company or service. We will never sell or give away your personal data or conversation history.” But there are important caveats to this claim.

The first is this: You can use Woebot via Facebook messenger or as a standalone iOS app. The Facebook version of Woebot shares conversations with Facebook itself.

Conversations with Woebot within Facebook Messenger are subject to the Facebook privacy policy. Facebook can see that you are talking to Woebot, and they can see the content of conversations.

The second is that Woebot can (like any company) update their privacy policy at any time.

Third: personal data they’ve gathered can be made available to any company with whom they merge.

Of course Woebot analyzes all your conversations with Woebot–it couldn’t claim to be doing AI for mental health otherwise.

You don’t have to create an account as such to use Woebot (or at least I didn’t at the time I started using the app), but it does ask for your phone number, which is a way you could be identified.

None of these issues particularly bother me, but I wouldn’t be surprised if your Woebot conversations end up later becoming fodder for targeting advertising.

Conclusion

To the extent that practicing CBT skills improves people’s mood, Woebot could be helpful. But to get through the difficulties in navigating capabilities and the distracting hedgehogs is pretty difficult. For more straightforward CBT coaching, I’d go with Moodnotes. I like Moodnotes’ enrichment exercises for when you’re feeling positive too.

One thought on “App Review: Woebot

  1. thanks interesting.
    I agree Woebot needs a ‘short-cut’ menu to various specific modules for users who know what they want.
    Given the recent funding, I hope Woebot’s AI conversationali and analytic capability will be improving fast.
    p.s. I have no connection

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s