DIRAC - Detection and Identification of Rare Audio-visual Cues

Czech title

Detekce a identifikace řídkých audiovizuálních podnětů - DIRAC

Type

grant

Keywords

audio, video, detection

Abstract

Today’s computers can do many amazing things but there are still many "trivial" but important tasks they cannot do well. In particular, current information extraction techniques perform well when event types are well represented in the training data but often fail when encountering information-rich unexpected rare events. DIRAC project addresses this crucial machine weakness and aims at designing and developing an environment-adaptive autonomous artificial cognitive system that will detect, identify and classify possibly threatening rare events from the information derived by multiple active information-seeking audio-visual sensors.

Biological organisms rely for their survival on detecting and identifying new events. DIRAC therefore strives to combine its expertise in physiology of mammalian auditory and visual cortex and in audio/visual recognition engineering with the aim to move the art of audiovisual machine recognition from the classical signal processing/pattern classification paradigm to human-like information extraction. This means, among other things, to move from interpretation of all incoming data to reliable rejection of non-informative inputs, from passive acquisition of a single incoming stream to active search for the most relevant information in multiple streams, and from a system optimized for one static environment to autonomous adaptation to new changing environments, thus forming foundation for a new generation of efficient cognitive information processing technologies.

DIRAC is an EU IP IST project of the 6th Framework Program. Its duration is 5 years, from January 2006 until December 2010.

Partners of the project comes from all over the world and are the following: Idiap Research Institute (coordinator), Eidgenossische Technische Hochschule Zuerich (CH), The Hebrew University of Jerusalem (IL), Czech Technical University (CS), Carl von Ossietzky Universitaet Oldenburg (DE), Leibniz Institute for Neurobiology (DE), Katholieke Universiteit Leuven (B), Oregon Health and Science University OGI School of Science and Engineerring (USA).

Team members

Heřmanský Hynek, prof. Ing., Dr.Eng. (UPGM FIT VUT) , research leader
Burget Lukáš, doc. Ing., Ph.D. (UPGM FIT VUT) , team leader
Hannemann Mirko, Dipl.-Ing. (UPGM FIT VUT) , team leader
Kombrink Stefan, Dipl.-Inf -Ling (UPGM FIT VUT) , team leader
Mikolov Tomáš, Ing. (UPGM FIT VUT) , team leader

Publications

2012

KOMBRINK Stefan, HANNEMANN Mirko and BURGET Lukáš. Out-of-Vocabulary Word Detection and Beyond. Detection and Identification of Rare Audiovisual Cues. Studies in Computational Intelligence, 384. Springer-Verlag Berlin Heidelberg: Springer Verlag, 2012, pp. 57-65. ISBN 978-3-642-24033-1. Detail

2011

MIKOLOV Tomáš, KOMBRINK Stefan, BURGET Lukáš, ČERNOCKÝ Jan and KHUDANPUR Sanjeev. Extensions of Recurrent Neural Network Language Model. In: Proceedings of the 2011 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2011. Praha: IEEE Signal Processing Society, 2011, pp. 5528-5531. ISBN 978-1-4577-0537-3. Detail
KOMBRINK Stefan and MIKOLOV Tomáš. Recurrent Neural Network Language Modeling Applied to the Brno AMI/AMIDA 2009 Meeting Recognizer Setup. In: Proceedings of the 17th Conference STUDENT EEICT 2011. Volume 3. Brno: Brno University of Technology, 2011, pp. 527-531. ISBN 978-80-214-4273-3. Detail
DEORAS Anoop, MIKOLOV Tomáš, KOMBRINK Stefan, KARAFIÁT Martin and KHUDANPUR Sanjeev. Variational Approximation of Long-span Language Models for LVCSR. In: Proceedings of the 2011 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2011. Praha: IEEE Signal Processing Society, 2011, pp. 5532-5535. ISBN 978-1-4577-0537-3. Detail

2010

KOMBRINK Stefan and HANNEMANN Mirko. Final system for identifying unexpected acoustic inputs (BUT). Brno: The Information Society Technologies (IST) 6th Framework programme, 2010. Detail
KOMBRINK Stefan, HANNEMANN Mirko and BURGET Lukáš. Out-of-vocabulary word detection and beyond. In: ECML PKDD 2010 Proceedings and Journal Content. Barcelona, 2010, pp. 1-8. Detail
KOMBRINK Stefan, HANNEMANN Mirko, BURGET Lukáš and HEŘMANSKÝ Hynek. Recovery of Rare Words in Lecture Speech. In: Proc. Text, Speech and Dialogue 2010. Brno: Springer Verlag, 2010, pp. 330-337. ISBN 978-3-642-15759-2. ISSN 0302-9743. Detail
MIKOLOV Tomáš, KARAFIÁT Martin, BURGET Lukáš, ČERNOCKÝ Jan and KHUDANPUR Sanjeev. Recurrent neural network based language model. In: Proceedings of the 11th Annual Conference of the International Speech Communication Association (INTERSPEECH 2010). Makuhari, Chiba: International Speech Communication Association, 2010, pp. 1045-1048. ISBN 978-1-61782-123-3. ISSN 1990-9772. Detail
HANNEMANN Mirko, KOMBRINK Stefan, KARAFIÁT Martin and BURGET Lukáš. Similarity Scoring for Recognizing Repeated Out-of-VocabularyWords. In: Proceedings of the 11th Annual Conference of the International Speech Communication Association (INTERSPEECH 2010). Makuhari, Chiba: International Speech Communication Association, 2010, pp. 897-900. ISBN 978-1-61782-123-3. ISSN 1990-9772. Detail
ČERNOCKÝ Jan, SZŐKE Igor, HANNEMANN Mirko and KOMBRINK Stefan. Word-subword based keyword spotting with implications in OOV detection. Pacific Grove: Institute of Electrical and Electronics Engineers, 2010. Detail

2009

BRÜMMER Niko, STRASHEIM Albert, HUBEIKA Valiantsina, MATĚJKA Pavel, BURGET Lukáš and GLEMBEK Ondřej. Discriminative Acoustic Language Recognition via Channel-Compensated GMM Statistics. In: Proc. Interspeech 2009. Brighton: International Speech Communication Association, 2009, pp. 2187-2190. ISBN 978-1-61567-692-7. ISSN 1990-9772. Detail
KOMBRINK Stefan, BURGET Lukáš, MATĚJKA Pavel, KARAFIÁT Martin and HEŘMANSKÝ Hynek. Posterior-based Out of Vocabulary Word Detection in Telephone Speech. In: Proc. Interspeech 2009. Brighton: International Speech Communication Association, 2009, pp. 80-83. ISSN 1990-9772. Detail

2008

BURGET Lukáš, SCHWARZ Petr, MATĚJKA Pavel, HANNEMANN Mirko, RASTROW Ariya, WHITE Christopher, KHUDANPUR Sanjeev, HEŘMANSKÝ Hynek and ČERNOCKÝ Jan. Combination of strongly and weakly constrained recognizers for reliable detection of OOVs. In: Proc. International Conference on Acoustics, Speech, and Signal Processing (ICASSP). Las Vegas: IEEE Signal Processing Society, 2008, p. 4. ISBN 1-4244-1484-9. Detail
BURGET Lukáš, BRÜMMER Niko, REYNOLDS Douglas, KENNY Patrick, PELECANOS Jason, VOGT Robbie, CASTALDO Fabio, DEHAK Najim, DEHAK Reda, GLEMBEK Ondřej, KARAM Zahi, NOECKER John Jr., NA Hye Young, COSTIN Ciprian C., HUBEIKA Valiantsina, KAJAREKAR Sachin, SCHEFFER Nicolas and ČERNOCKÝ Jan. Robust Speaker Recognition Over Varying Channels. Baltimore: Johns Hopkins University, 2008. Detail

Products

2009

Out-of-vocabulary Words Detection Demonstrator, software, 2009
Authors: Kombrink Stefan, Hannemann Mirko, Burget Lukáš, Heřmanský Hynek Detail
Web-based lecture browser, software, 2009
Authors: Žižka Josef, Fapšo Michal, Szőke Igor Detail