Project Details

ESPERANTO - Exchanges for SPEech ReseArch aNd TechnOlogies

Project Period: 1. 1. 2021 - 31. 12. 2025

Project Type: grant

Code: 101007666

Agency: European Comission EU

Program: Horizon 2020

Czech title
Výměny pro výzkum řeči a technologií
Type
grant
Keywords

artificial intelligence, intelligent systems, multi agent systems, machine learning, data mining, statistical data processing and application, modelling engineering, human computer interaction, natural language processing, speech processing, neural networks, explainability, human assisted learning, low resources, natural language processing, standardization, evaluation

Abstract

The ESPERANTO project aims at pushing speech processing technologies to their next step in order to enable the diffusion of these technologies in European SMEs and to maximize and securize their use in the civil society for forensic, health or education. The ESPERANTO consortium forsees that the next generation of artificial intelligence algorithms for speech processing should : 1. be more accessible : via a larger number of spoken languages, and for applications where resources are strongly limited (health, education, robotics); 2. integrate a human in the loop to guaranty a higher usability and ease of deployment and maintenance; 3. be explainable in order to enable sensitive applications related to forensic or health and contribute to personal data preservation by detecting and characterizing existing biases due to the data-driven nature of current speech technologies. ESPERANTO intends to lead the scientific community by releasing evaluation metrics, protocols and standards that will boost the development and evaluation of this new generation of algorithms. To achieve this ambitious goal, the ESPERANTO project gathers a large and trans-sectorial community of experts in speech related applications such as speech transcription, separation, enhancement, translation, understanding and speaker recognition and diarization to transfer knowledge, organize, produce and standardize resources with the aim of catalyzing and cross-pollenizing this area. The main goals of the ESPERANTO project are: - support the development of open-source tools that will encourage fast developement, exchanges and reproducibility; - produce tutorials and competitive baselines on various topics of speech processing in order to boost the fostering of new speech-AI students, researchers and engineers; - facilitate the collection and sharing of linguistic and speech resources through standards; - organize workshops to progress on the speech technologies and favor tranfer of knowledge.

Team members
Kudla Radim, Ing. (UPGM FIT VUT) , research leader
Burget Lukáš, doc. Ing., Ph.D. (UPGM FIT VUT) , team leader
Plchot Oldřich, Ing., Ph.D. (UPGM FIT VUT) , team leader
Rohdin Johan A., Dr. (UPGM FIT VUT) , team leader
Kohlová Renata, Ing. (UPGM FIT VUT)
Landini Federico Nicolás (UPGM FIT VUT)
Matějka Pavel, Ing., Ph.D. (UPGM FIT VUT)
Mošner Ladislav, Ing. (UPGM FIT VUT)
Silnova Anna, MSc., Ph.D. (UPGM FIT VUT)
Publications

2024

  • HAN Jiangyu, LANDINI Federico Nicolás, ROHDIN Johan A., DIEZ Sánchez Mireia, BURGET Lukáš, CAO Yuhang, LU Heng and ČERNOCKÝ Jan. Diacorrect: Error Correction Back-End for Speaker Diarization. In: ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Seoul: IEEE Signal Processing Society, 2024, pp. 11181-11185. ISBN 979-8-3503-4485-1. Detail
  • LANDINI Federico Nicolás, DIEZ Sánchez Mireia, STAFYLAKIS Themos and BURGET Lukáš. DiaPer: End-to-End Neural Diarization With Perceiver-Based Attractors. IEEE Transactions on Audio, Speech, and Language Processing, vol. 32, no. 7, 2024, pp. 3450-3465. ISSN 1558-7916. Detail
  • KLEMENT Dominik, DIEZ Sánchez Mireia, LANDINI Federico Nicolás, BURGET Lukáš, SILNOVA Anna, DELCROIX Marc and TAWARA Naohiro. Discriminative Training of VBx Diarization. In: ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings. Seoul: IEEE Signal Processing Society, 2024, pp. 11871-11875. ISBN 979-8-3503-4485-1. Detail
  • ZHANG Lin, STAFYLAKIS Themos, LANDINI Federico Nicolás, DIEZ Sánchez Mireia, SILNOVA Anna and BURGET Lukáš. Do End-to-End Neural Diarization Attractors Need to Encode Speaker Characteristic Information?. In: Proceedings of Odyssey 2024: The Speaker and Language Recognition Workshop. Québec City: International Speech Communication Association, 2024, pp. 123-130. Detail
  • STAFYLAKIS Themos, SILNOVA Anna, ROHDIN Johan A., PLCHOT Oldřich and BURGET Lukáš. Challenging margin-based speaker embedding extractors by using the variational information bottleneck. In: Proceedings of Interspeech 2024. Kos: International Speech Communication Association, 2024, pp. 3220-3224. ISSN 1990-9772. Detail

2023

  • SILNOVA Anna, SLAVÍČEK Josef, MOŠNER Ladislav, KLČO Michal, PLCHOT Oldřich, MATĚJKA Pavel, PENG Junyi, STAFYLAKIS Themos and BURGET Lukáš. ABC System Description for NIST LRE 2022. In: Proceedings of NIST LRE 2022 Workshop. Washington DC: National Institute of Standards and Technology, 2023, pp. 1-5. Detail
  • PENG Junyi, PLCHOT Oldřich, STAFYLAKIS Themos, MOŠNER Ladislav, BURGET Lukáš and ČERNOCKÝ Jan. An attention-based backend allowing efficient fine-tuning of transformer models for speaker verification. In: 2022 IEEE Spoken Language Technology Workshop, SLT 2022 - Proceedings. Doha: IEEE Signal Processing Society, 2023, pp. 555-562. ISBN 978-1-6654-7189-3. Detail
  • KESIRAJU Santosh, BENEŠ Karel, TIKHONOV Maksim and ČERNOCKÝ Jan. BUT Systems for IWSLT 2023 Marathi - Hindi Low Resource Speech Translation Task. In: 20th International Conference on Spoken Language Translation, IWSLT 2023 - Proceedings of the Conference. Toronto (in-person and online): Association for Computational Linguistics, 2023, pp. 227-234. ISBN 978-1-959429-84-5. Detail
  • MATĚJKA Pavel, SILNOVA Anna, SLAVÍČEK Josef, MOŠNER Ladislav, PLCHOT Oldřich, KLČO Michal, PENG Junyi, STAFYLAKIS Themos and BURGET Lukáš. Description and Analysis of ABC Submission to NIST LRE 2022. In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. Dublin: International Speech Communication Association, 2023, pp. 511-515. ISSN 1990-9772. Detail
  • STAFYLAKIS Themos, MOŠNER Ladislav, KAKOUROS Sofoklis, PLCHOT Oldřich, BURGET Lukáš and ČERNOCKÝ Jan. Extracting speaker and emotion information from self-supervised speech models via channel-wise correlations. In: 2022 IEEE Spoken Language Technology Workshop, SLT 2022 - Proceedings. Doha: IEEE Signal Processing Society, 2023, pp. 1136-1143. ISBN 978-1-6654-7189-3. Detail
  • PENG Junyi, PLCHOT Oldřich, STAFYLAKIS Themos, MOŠNER Ladislav, BURGET Lukáš and ČERNOCKÝ Jan. Improving Speaker Verification with Self-Pretrained Transformer Models. In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. Dublin: International Speech Communication Association, 2023, pp. 5361-5365. ISSN 1990-9772. Detail
  • MOŠNER Ladislav, PLCHOT Oldřich, PENG Junyi, BURGET Lukáš and ČERNOCKÝ Jan. Multi-Channel Speech Separation with Cross-Attention and Beamforming. In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. Dublin: International Speech Communication Association, 2023, pp. 1693-1697. ISSN 1990-9772. Detail
  • LANDINI Federico Nicolás, DIEZ Sánchez Mireia, LOZANO Díez Alicia and BURGET Lukáš. Multi-Speaker and Wide-Band Simulated Conversations as Training Data for End-to-End Neural Diarization. In: Proceedings of ICASSP 2023. Rhodes Island: IEEE Signal Processing Society, 2023, pp. 1-5. ISBN 978-1-7281-6327-7. Detail
  • PENG Junyi, STAFYLAKIS Themos, GU Rongzhi, PLCHOT Oldřich, MOŠNER Ladislav, BURGET Lukáš and ČERNOCKÝ Jan. Parameter-Efficient Transfer Learning of Pre-Trained Transformer Models for Speaker Verification Using Adapters. In: ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings. Rhodes Island: IEEE Signal Processing Society, 2023, pp. 1-5. ISBN 978-1-7281-6327-7. Detail
  • KAKOUROS Sofoklis, STAFYLAKIS Themos, MOŠNER Ladislav and BURGET Lukáš. Speech-Based Emotion Recognition with Self-Supervised Models Using Attentive Channel-Wise Correlations and Label Smoothing. In: Proceedings of ICASSP 2023. Rhodes Island: IEEE Signal Processing Society, 2023, pp. 1-5. ISBN 978-1-7281-6327-7. Detail
  • KESIRAJU Santosh, SARVAŠ Marek, PAVLÍČEK Tomáš, MACAIRE Cécile and CIUBA Alejandro. Strategies for Improving Low Resource Speech to Text Translation Relying on Pre-trained ASR Models. In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. Dublin: International Speech Communication Association, 2023, pp. 2148-2152. ISSN 1990-9772. Detail
  • SILNOVA Anna, BRUMMER Johan Nikolaas Langenhoven, SWART Albert du Preez and BURGET Lukáš. Toroidal Probabilistic Spherical Discriminant Analysis. In: Proceedings of ICASSP 2023. Rhodes Island: IEEE Signal Processing Society, 2023, pp. 1-5. ISBN 978-1-7281-6327-7. Detail

2022

  • SILNOVA Anna, STAFYLAKIS Themos, MOŠNER Ladislav, PLCHOT Oldřich, ROHDIN Johan A., MATĚJKA Pavel, BURGET Lukáš, GLEMBEK Ondřej and BRUMMER Johan Nikolaas Langenhoven. Analyzing speaker verification embedding extractors and back-ends under language and channel mismatch. In: Proceedings of The Speaker and Language Recognition Workshop (Odyssey 2022). Beijing: International Speech Communication Association, 2022, pp. 9-16. Detail
  • KOCOUR Martin, UMESH Jahnavi, KARAFIÁT Martin, ŠVEC Ján, LOPEZ Fernando, BENEŠ Karel, DIEZ Sánchez Mireia, SZŐKE Igor, LUQUE Jordi, VESELÝ Karel, BURGET Lukáš and ČERNOCKÝ Jan. BCN2BRNO: ASR System Fusion for Albayzin 2022 Speech to Text Challenge. In: Proceedings of IberSpeech 2022. Granada: International Speech Communication Association, 2022, pp. 276-280. Detail
  • ALAM Jahangir, BURGET Lukáš, GLEMBEK Ondřej, MATĚJKA Pavel, MOŠNER Ladislav, PLCHOT Oldřich, ROHDIN Johan A., SILNOVA Anna and STAFYLAKIS Themos et al. Development of ABC systems for the 2021 edition of NIST Speaker Recognition evaluation. In: Proceedings of The Speaker and Language Recognition Workshop (Odyssey 2022). Beijing: International Speech Communication Association, 2022, pp. 346-353. Detail
  • PENG Junyi, GU Rongzhi, MOŠNER Ladislav, PLCHOT Oldřich, BURGET Lukáš and ČERNOCKÝ Jan. Learnable Sparse Filterbank for Speaker Verification. In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. Incheon: International Speech Communication Association, 2022, pp. 5110-5114. ISSN 1990-9772. Detail
  • MOŠNER Ladislav, PLCHOT Oldřich, BURGET Lukáš and ČERNOCKÝ Jan. Multi-Channel Speaker Verification with Conv-Tasnet Based Beamformer. In: ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings. Singapore: IEEE Signal Processing Society, 2022, pp. 7982-7986. ISBN 978-1-6654-0540-9. Detail
  • MOŠNER Ladislav, PLCHOT Oldřich, BURGET Lukáš and ČERNOCKÝ Jan. Multisv: Dataset for Far-Field Multi-Channel Speaker Verification. In: ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings. Singapore: IEEE Signal Processing Society, 2022, pp. 7977-7981. ISBN 978-1-6654-0540-9. Detail
  • BRUMMER Johan Nikolaas Langenhoven, SWART Albert du Preez, MOŠNER Ladislav, SILNOVA Anna, PLCHOT Oldřich, STAFYLAKIS Themos and BURGET Lukáš. Probabilistic Spherical Discriminant Analysis: An Alternative to PLDA for length-normalized embeddings. In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. Incheon: International Speech Communication Association, 2022, pp. 1446-1450. ISSN 1990-9772. Detail
  • STAFYLAKIS Themos, MOŠNER Ladislav, PLCHOT Oldřich, ROHDIN Johan A., SILNOVA Anna, BURGET Lukáš and ČERNOCKÝ Jan. Training Speaker Embedding Extractors Using Multi-Speaker Audio with Unknown Speaker Boundaries. In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. Incheon: International Speech Communication Association, 2022, pp. 605-609. ISSN 1990-9772. Detail

2021

  • STAFYLAKIS Themos, ROHDIN Johan A. and BURGET Lukáš. Speaker embeddings by modeling channel-wise correlations. In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. Brno: International Speech Communication Association, 2021, pp. 501-505. ISSN 1990-9772. Detail
Back to top