Depression may live in the brain, but scientists have developed a new smartphone app to detect the disorder by looking for clues on your face.
MoodCapture uses AI to assess micro-changes to a person’s face – such as their gaze, eye movement, and how the person tilted their head – to determine whether they were depressed.
The app, which was funded by the National Institute of Health, takes pictures with the front-facing camera and sends an alert if it identified a trend in facial expressions by looking at the position of the participants’ lips, eyes and the depression lines in their face.
According to the study, MoodCapture was correct in identifying people with depression 75 percent of the time.
MoodCapture identified if participant’s had depressive symptoms based on their facial features, lighting, and background objects
About eight percent of U.S. adults are diagnosed with depression each year, amounting to roughly 21 million Americans
More research still needs to be conducted, but researchers said MoodCapture could be made available to the public as early as within five years.
‘This is the first time that natural ‘in-the-wild’ images have been used to predict depression,’ said Andrew Campbell, the study’s lead author and computer science professor at Dartmouth College.
‘People use facial recognition software to unlock their phones hundreds of times a day,’ he said.
‘MoodCapture uses a similar technology pipeline of facial recognition technology with deep learning and AI hardware, so there is terrific potential to scale up this technology without any additional input or burden on the user.’
The study broke down participant’s facial expressions by looking at the angle of their features such as the way their eyebrows creased, how they tilted their heads and whether their lips were angled up or down.
Over time, the app noticed patterns that were specific to the user and correctly identified users who often had a flat expression – their features didn’t change – and were in a dimly lit room for an extended period of time as having depression.
An estimated 60 percent of people diagnosed with depression don’t seek help
Researchers recruited 181 participants in the U.S. who had major depressive disorder according to a questionnaire they were asked to fill out.
The participants were given three surveys daily to assess their moods and MoodCapture discreetly took up to five photos when participants answered a specific question like ‘I have felt down, depressed, or hopeless,’ to see if it could correctly identify that feeling.
‘We chose this question as we believed it would best capture participants’ genuine emotions related to depression,’ the study said.
The images were taken randomly with the front-facing camera over 90 days, and looked at specific facial expressions of 177 participants including gaze, eye movement, lighting, how the person positioned their head, and others.
The researchers collected a total of 125,335 images, but omitted 15,063 photos that were too blurry, didn’t show a face, featured children, or contained nudity.
MoodCapture looked at the dominant colors in the participant’s environment, the lighting condition, where the photo was taken – whether it was indoors or outdoors – any background objects that could be used to gauge the user’s activities, and the number of people in the image.
Such details, like a dim lighting, can reveal insights into a person’s mental state.
MoodCapture can the sequence images in real-time, combining facial features and background information to predict the severity of their depression.
Researchers asked a series of questions to determine if the person had depressive symtpoms and connected it to MoodCapture’s findings
MoodCapture not only identified if the participant was experiencing depressive symptoms but also suggested preventative measures like going outside or reaching out to a friend for help.
‘Telling someone something bad is going on with them has the potential to make things worse,’ said Nicholas Jacobson, the study’s co-author and assistant professor of biomedical data science and psychiatry at Dartmouth’s Center for Technology and Behavioral Health.
‘We think that MoodCapture opens the door to assessment tools that would help detect depression in the moments before it gets worse,’ Jacobson said.
Major depression affects more than eight percent of American adults every year, amounting to an estimated 21 million people, but an estimated 60 percent of people who are diagnosed with depression don’t seek professional support, according to Healthline.
Researchers said the study’s results were promising, and while more research needs to be conducted, Campbell said they estimate this technology could be available in the next five years, adding: ‘We’ve shown this is doable.
‘This demonstrates a path toward a powerful tool for evaluating a person’s mood in a passive way and using the data as a basis for therapeutic intervention.’
However, researchers advised that MoodCapture, and any other applications like it, should not be used alone and should instead be paired with other interventions for people with depression.
‘Our goal is to capture the changes in symptoms that people with depression experience in their daily lives,’ Jacobson said.
‘If we can use this to predict and understand the rapid changes in depression symptoms, we can ultimately head them off and treat them.
‘The more in the moment we can be, the less profound the impact of depression will be.’
