Publication Details
MixedEmotions: An Open-Source Toolbox for Multimodal Emotion Analysis
Wood Ian (NUIG)
Negi Sapna (NUIG)
Arcan Mihael (NUIG)
McCrae John P. (NUIG)
Abele Andrejs (NUIG)
Robin Cécile (NUIG)
Andryushechkin Vladimir (NUIG)
Ziad Housam (NUIG)
Sagha Hesam (UNIPAS)
Schmitt Maxmilian (UNIPAS)
Schuller Björn W. (UNIPAS)
Sánchez-Rada J. Fernando (UPN)
Iglesias Carlos A. (UPN)
Navarro Carlos (PDDG)
Giefer Andreas (DW)
Heise Nicolaus (DW)
Masucci Vincenzo (ESM)
Danza Francesco A. (ESM)
Caterino Ciro (ESM)
Smrž Pavel, doc. RNDr., Ph.D. (DCGM FIT BUT)
Hradiš Michal, Ing., Ph.D. (DCGM FIT BUT)
Povolný Filip, Ing. (Phonexia)
Klimeš Marek, Ing. (Phonexia)
Matějka Pavel, Ing., Ph.D. (Phonexia)
Tummarello Giovanni (SIRSOL)
emotion analysis, open source toolbox, affective computing, linked data, audio processing, text processing, video processing
Recently, there is an increasing tendency to embed functionalities for recognizing emotions from user-generated media content in automated systems such as call-centre operations, recommendations, and assistive technologies, providing richer and more informative user and content profiles. However, to date, adding these functionalities was a tedious, costly, and time-consuming effort, requiring identification and integration of diverse tools with diverse interfaces as required by the use case at hand. The MixedEmotions Toolbox leverages the need for such functionalities by providing tools for text, audio, video, and linked data processing within an easily integrable plug-and-play platform. These functionalities include: 1) for text processing: emotion and sentiment recognition; 2) for audio processing: emotion, age, and gender recognition; 3) for video processing: face detection and tracking, emotion recognition, facial landmark localization, head pose estimation, face alignment, and body pose estimation; and 4) for linked data: knowledge graph integration. Moreover, the MixedEmotions Toolbox is open-source and free. In this paper, we present this toolbox in the context of the existing landscape, and provide a range of detailed benchmarks on standard test-beds showing its state-of-the-art performance. Furthermore, three real-world use cases show its effectiveness, namely, emotion-driven smart TV, call center monitoring, and brand reputation analysis.
@ARTICLE{FITPUB11815,
author = "Paul Buitelaar and Ian Wood and Sapna Negi and Mihael Arcan and P. John McCrae and Andrejs Abele and C\'{e}cile Robin and Vladimir Andryushechkin and Housam Ziad and Hesam Sagha and Maxmilian Schmitt and W. Bj{\"{o}}rn Schuller and Fernando J. S\'{a}nchez-Rada and A. Carlos Iglesias and Carlos Navarro and Andreas Giefer and Nicolaus Heise and Vincenzo Masucci and A. Francesco Danza and Ciro Caterino and Pavel Smr\v{z} and Michal Hradi\v{s} and Filip Povoln\'{y} and Marek Klime\v{s} and Pavel Mat\v{e}jka and Giovanni Tummarello",
title = "MixedEmotions: An Open-Source Toolbox for Multimodal Emotion Analysis",
pages = "2454--2465",
journal = "IEEE Transactions on Multimedia",
volume = 20,
number = 9,
year = 2018,
ISSN = "1520-9210",
doi = "10.1109/TMM.2018.2798287",
language = "english",
url = "https://www.fit.vut.cz/research/publication/11815"
}