People with complex needs may not have reliable facial expressions or body language due to the severity of their condition. When we don’t understand a person’s attempt to communicate it can lead to feelings of frustration, anger and isolation. This often contributes to low mood and frustration that may result in challenging behaviours such as aggression, self-harm and falls. There are also individuals who can communicate but experience episodes when they cannot express themselves in words; for example, when a person with autism experiences temporary sensory overload.

Many non-verbal people do not have the capacity to use sign language or standard assistive technologies. It is often the case that a primary carer or key worker is the only one who understands them. Empathic addresses this problem by enabling everyone to interpret up to 10 emotions; even if they have never met the non-verbal person before.

In 2019 we began working with families and health care providers to gather audio data from non-verbal individuals in real world settings. We measured sound waves and identified the characteristics specific to 10 emotions. Our research supports existing academic findings that show consistency across languages; our tests include English, Portuguese and Turkish. We also found significant similarities across diverse conditions such as autism, dementia and brain injury.

Extensive testing shows that using Empathic can reduce frustration and enhance mood. For example, we asked users to share the app’s interpretations with the non-verbal person they were supporting. They usually said something like “I see you are getting frustrated, let’s do something else” or “It shows me that you are happy, we’ll keep playing this music.” Users tell us that using Empathic helped to decrease frustration, improve mood and enhance interactions. Some users reported that the supported person became more sociable and increased their efforts to vocalise; as one mother put it “when she realised we were listening, she tried to talk to us more often, in her own way”.

We are about to launch Empathic for healthcare and home care providers. This offers a multiuser dashboard, reports and real time updates that can be accessed on any smart device. We are also working on hands free options for use in smart homes and sensory rooms.