Publication Details
Revisiting joint decoding based multi-talker speech recognition with DNN acoustic model
Žmolíková Kateřina, Ing., Ph.D. (FIT)
ONDEL YANG, L.
Švec Ján, Ing. (DCGM)
Delcroix Marc
OCHIAI, T.
Burget Lukáš, doc. Ing., Ph.D. (DCGM)
Černocký Jan, prof. Dr. Ing. (DCGM)
Multi-talker speech recognition, Permutation invariant
training, Factorial Hidden Markov models
In typical multi-talker speech recognition systems, a neural
network-based acoustic model predicts senone state posteriors
for each speaker. These are later used by a single-talker decoder
which is applied on each speaker-specific output stream separately.
In this work, we argue that such a scheme is sub-optimal
and propose a principled solution that decodes all speakers
jointly. We modify the acoustic model to predict joint state
posteriors for all speakers, enabling the network to express uncertainty
about the attribution of parts of the speech signal to
the speakers. We employ a joint decoder that can make use
of this uncertainty together with higher-level language information.
For this, we revisit decoding algorithms used in factorial
generative models in early multi-talker speech recognition systems.
In contrast with these early works, we replace the GMM
acoustic model with DNN, which provides greater modeling
power and simplifies part of the inference. We demonstrate the
advantage of joint decoding in proof of concept experiments on
a mixed-TIDIGITS dataset.
@inproceedings{BUT179827,
author="KOCOUR, M. and ŽMOLÍKOVÁ, K. and ONDEL YANG, L. and ŠVEC, J. and DELCROIX, M. and OCHIAI, T. and BURGET, L. and ČERNOCKÝ, J.",
title="Revisiting joint decoding based multi-talker speech recognition with DNN acoustic model",
booktitle="Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH",
year="2022",
journal="Proceedings of Interspeech",
number="9",
pages="4955--4959",
publisher="International Speech Communication Association",
address="Incheon",
doi="10.21437/Interspeech.2022-10406",
issn="1990-9772",
url="https://www.isca-speech.org/archive/pdfs/interspeech_2022/kocour22_interspeech.pdf"
}