Scientists from FIT co-operate on development of an app aiming to make psychological care more effective

To develop a mobile app capable of recognising mental condition based on voice and image and, at the same time, enabling physicians to be in touch with their patients more frequently - for the past several months, this has been the goal of scientists from the Faculty of Information Technology of BUT co-operating with other foreign experts on the EcoWeB (Emotional Competence Well-Being) EU project. Its aim is to utilise the potential offered by mobile technologies to make psychological care more accessible to young people.

To develop a mobile app capable of recognising mental condition based on voice and image and, at the same time, enabling physicians to be in touch with their patients more frequently - for the past several months, this has been the goal of scientists from the Faculty of Information Technology of BUT co-operating with other foreign experts on the EcoWeB (Emotional Competence Well-Being) EU project. Its aim is to utilise the potential offered by mobile technologies to make psychological care more accessible to young people.

The mobile app should make psychological care for teenagers and young people more effective and intensive and, at the same time, it should serve as a prevention of occurrence of mental disorders. "People are often coming to scheduled check-ups with their psychologists when it is not necessary and when they are not aware of any acute issues. At other times, they would need more frequent consultations, but that is not possible as the psychologist just cannot schedule any more people for the given day. A video call cannot replace meeting in person, but it can help in certain situations. Furthermore, the psychologist will also gain the advantage of seeing the patient's environment and mental condition," described Pavel Smrž, who co-operates on the development together with his other FIT BUT colleagues.

Apart from arranging video calls between physicians and patients, the app also motivates its users to regularly record their feelings and mood. It can also evaluate the emotional state of a patient based on his/her speech. If the patient grants his/her consent, the app can monitor the non-verbal attributes of speech, such as voice frequency and dynamics, even during normal usage of the phone. Thanks to this, it can recognise if a person has been upset or emotionally unstable during the day. A psychologist can then get a more comprehensive overview of the patient's state and the development of his/her condition and has the option to adjust therapy accordingly.

While experts from the German company Audeering focus on analysis of speech within the ECoWeB project, experts from FIT BUT focus on recognition of emotions from video recordings. "We have plenty of experience from previous projects in which we focused on recognition of human emotions from a video recording using machine learning. Based on micro-movements of the facial muscles - mouth, eyes, eyebrows or lips - our algorithm is able to identify the current emotional state of person or even if the person is saying something that is at variance with his/her attitude and feelings based on micro-movements of the facial muscles," added Smrž.

According to Pavel Smrž, the aim of the app is not to replace personal contact between a patient and a physician but to use mobile technology to make the contact even deeper. "Our motivation is to guide the expert's work so that he/she has information about the patient even when they lack personal contact. Thanks to voice or image-based detection, the app can notify the physician that the patient has recently experienced emotional shock; the physician can then ask about that," he clarified. Another possible advantage is making psychological care more accessible and affordable so that it can also focus on the much-needed prevention of occurrence of mental disorders in young people.

Development of the app should be completed by the end of April. The app will be subsequently tested by teams of psychologists on nearly a thousand volunteers from Germany, Belgium, United Kingdom, Spain, Greece, Denmark and Switzerland. Following the testing, it should be available to psychologists and their patients.