Project Details
MixedEmotions - Social Semantic Emotion Analysis for Innovative Multilingual Big Data Analytics Markets
Project Period: 1. 4. 2015 - 31. 3. 2017
Project Type: grant
Code: 644632
Agency: European Comission EU
Program: Horizon 2020
social semantics, multilingual multi-modal emotion analysis, knowledge graphs, Social TV, Brand Reputation Management, Call Centre Operations
Emotion analysis is central to tracking customer and user behaviour and satisfaction, which can be observed from user interaction in the form of explicit feedback through email, call center interaction, social media comments, etc., as well as implicit acknowledgement of approval or rejection through facial expressions, speech or other non-verbal feedback. In Europe specifically, but increasingly also globally, an added factor here is that user feedback can be in multiple languages, in text as well as in speech and audio-visual content. This implies different cultural backgrounds and thus different ways to produce and perceive emotions in everyday interactions, beyond the fact of having specific rules for encoding and decoding emotions in each language. Making sense of accumulated user interaction from different data sources, modalities and languages is challenging and has not yet been explored in fullness in an industrial context. Commercial solutions exist but do not address the multilingual aspect in a robust and large-scale setting and do not scale up to huge data volumes that need to be processed, or the integration of emotion analysis observations across data sources and/or modalities on a meaningful level, i.e. keeping track of entities involved as well the connections between them (who said what? to whom? in the context of which event, product, service?) In MixedEmotions we will implement an integrated Big Linked Data platform for emotion analysis across heterogeneous data sources, languages and modalities, building on existing state of the art tools, services and approaches that will enable the tracking of emotional aspects of user interaction and feedback on an entity level. The MixedEmotions platform will provide an integrated solution for Large-scale emotion analysis and fusion on heterogeneous, multilingual, text, speech, video and social media data streams, leveraging open access and proprietary data sources, exploiting also social context by leveraging social network graphs 􀆔 Semantic-level emotion information aggregation and integration through robust extraction of social semantic knowledge graphs for emotion analysis along multidimensional clusters The platform will be developed and evaluated in the context of three cross-domain Pilot Projects that are representative of a variety of data analytics markets: Social TV, Brand Reputation Management, Call Centre Operations. Each of the companies involved in the pilot projects have specific innovation objectives
Zemčík Pavel, prof. Dr. Ing. (UPGM FIT VUT) , team leader
Černocký Jan, prof. Dr. Ing. (UPGM FIT VUT)
Dytrych Jaroslav, Ing., Ph.D. (UPGM FIT VUT)
Matějka Jiří, Ing. (FIT VUT)
Nedeljković Sava, Bc. (FIT VUT)
Otrusina Lubomír, Ing. (UPGM FIT VUT)
Prexta Dávid, Bc. (FIT VUT)
Rusiňák Petr, Ing. (FIT VUT)
Suchánek Jan, Ing. (FIT VUT)
Švaňa Miloš, Bc. (FIT VUT)
Zapletal Jakub, Ing. (FIT VUT)
Zárybnický Jakub, Ing. (FIT VUT)
2018
- DYTRYCH Jaroslav and SMRŽ Pavel. Advanced User Interfaces for Semantic Annotation of Complex Relations in Text. In: Agents and Artificial Intelligence. Lecture Notes in Computer Science, vol. 10839. Cham: Springer International Publishing, 2018, pp. 205-221. ISBN 978-3-319-93581-2. ISSN 0302-9743. Detail
- BUITELAAR Paul, WOOD Ian, NEGI Sapna, ARCAN Mihael, MCCRAE John P., ABELE Andrejs, ROBIN Cécile, ANDRYUSHECHKIN Vladimir, ZIAD Housam, SAGHA Hesam, SCHMITT Maxmilian, SCHULLER Björn W., SÁNCHEZ-RADA J. Fernando, IGLESIAS Carlos A., NAVARRO Carlos, GIEFER Andreas, HEISE Nicolaus, MASUCCI Vincenzo, DANZA Francesco A., CATERINO Ciro, SMRŽ Pavel, HRADIŠ Michal, POVOLNÝ Filip, KLIMEŠ Marek, MATĚJKA Pavel and TUMMARELLO Giovanni. MixedEmotions: An Open-Source Toolbox for Multimodal Emotion Analysis. IEEE Transactions on Multimedia, vol. 20, no. 9, 2018, pp. 2454-2465. ISSN 1520-9210. Detail
2016
- POLOK Lukáš, ILA Viorela S. and SMRŽ Pavel. 3D Reconstruction Quality Analysis and Its Acceleration on GPU Clusters. In: Proceedings of European Signal Processing Conference 2016. Budapest: Institute of Electrical and Electronics Engineers, 2016, pp. 1108-1112. ISBN 978-0-9928626-6-4. Detail
- MACHÁČEK Jakub. BUTknot at SemEval-2016 task 5: Supervised machine learning with term substitution approach in Aspect Category Detection. In: SemEval 2016 - 10th International Workshop on Semantic Evaluation. San Diego: Association for Computational Linguistics, 2016, pp. 301-305. ISBN 978-1-941643-95-2. Detail
- SAGHA Hesam, MATĚJKA Pavel, GAVRYUOKOVA Maryna, POVOLNÝ Filip, MARCHI Erik and SCHULLER Björn W. Enhancing multilingual recognition of emotion in speech by language identification. In: 17TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION - Proceedings (INTERSPEECH 2016). San Francisco: International Speech Communication Association, 2016, pp. 2949-2953. ISSN 1990-9772. Detail
- POLOK Lukáš and SMRŽ Pavel. Increasing Double Precision Throughput on NVIDIA Maxwell GPUs. In: Proceedings of the 24th High Performance Computing Symposium. Pasadena / Los Angeles: Association for Computing Machinery, 2016, pp. 146-153. ISBN 978-1-5108-2318-1. Detail
- DYTRYCH Jaroslav and SMRŽ Pavel. Interaction Patterns in Computer-assisted Semantic Annotation of Text - An Empirical Evaluation. In: Proceedings of the 8th International Conference on Agents and Artificial Intelligence. Volume 2: ICAART. Setúbal: SciTePress - Science and Technology Publications, 2016, pp. 74-84. ISBN 978-989-758-172-4. Detail
- POPKOVÁ Anna, POVOLNÝ Filip, MATĚJKA Pavel, GLEMBEK Ondřej, GRÉZL František and ČERNOCKÝ Jan. Investigation of Bottle-Neck Features for Emotion Recognition. In: 19th International Conference, TSD 2016, Brno , Czech Republic, September 12-16, 2016, Proceedings. Lecture Notes in Computer Science, Lecture Notes in Artificial Intelligence, vol. 9924. Brno: International Speech Communication Association, 2016, pp. 426-434. ISSN 0302-9743. Detail
- POVOLNÝ Filip, MATĚJKA Pavel, HRADIŠ Michal, POPKOVÁ Anna, OTRUSINA Lubomír, SMRŽ Pavel, WOOD Ian, ROBIN Cécile and LAMEL Lori. Multimodal Emotion Recognition for AVEC 2016 Challenge. In: AVEC '16 Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge. Amsterdam: Association for Computing Machinery, 2016, pp. 75-82. ISBN 978-1-4503-4516-3. Detail
- OTRUSINA Lubomír and SMRŽ Pavel. WTF-LOD - A New Resource for Large-Scale NER Evaluation. In: Proceedings of the Tenth conference on International Language Resources and Evaluation (LREC'16). Portorož: European Language Resources Association, 2016, pp. 3299-3302. ISBN 978-2-9517408-9-1. Detail
2016
- Video emotion web service, software, 2016
Authors: Hradiš Michal, Kohút Jan Detail
2015
- Blogs downloader for MixedEmotions project, software, 2015
Authors: Otrusina Lubomír, Smrž Pavel Detail - Corpora Processing Software, software, 2015
Authors: Doležal Jan, Dytrych Jaroslav, Karásek Miroslav, Kouřil Jan, Otrusina Lubomír, Smrž Pavel Detail - Semantic Enrichment Component, software, 2015
Authors: Doležal Jan, Dytrych Jaroslav, Kouřil Jan, Otrusina Lubomír, Smrž Pavel Detail