Publication Details

Semi-supervised Sequence-to-sequence ASR using Unpaired Speech and Text

BASKAR Murali K., WATANABE Shinji, ASTUDILLO Ramon, HORI Takaaki, BURGET Lukáš and ČERNOCKÝ Jan. Semi-supervised Sequence-to-sequence ASR using Unpaired Speech and Text. In: Proceedings of Interspeech. Graz: International Speech Communication Association, 2019, pp. 3790-3794. ISSN 1990-9772. Available from: https://www.isca-speech.org/archive/Interspeech_2019/pdfs/3167.pdf
Czech title
ASR založené na převodu sekvencí na sekvence s lehkou supervizí využívající nesouvisející řečová a textová data
Type
conference paper
Language
english
Authors
Baskar Murali K. (DCGM FIT BUT)
Watanabe Shinji, Dr. (JHU)
Astudillo Ramon (IBM Watson)
Hori Takaaki (MERL)
Burget Lukáš, doc. Ing., Ph.D. (DCGM FIT BUT)
Černocký Jan, prof. Dr. Ing. (DCGM FIT BUT)
URL
Keywords

Sequence-to-sequence, end-to-end, ASR, TTS, semi-supervised, unsupervised, cycle consistency

Abstract

Sequence-to-sequence automatic speech recognition (ASR) models require large quantities of data to attain high performance. For this reason, there has been a recent surge in interest for unsupervised and semi-supervised training in such models. This work builds upon recent results showing notable improvements in semi-supervised training using cycle-consistency and related techniques. Such techniques derive training procedures and losses able to leverage unpaired speech and/or text data by combining ASR with Text-to-Speech (TTS) models. In particular, this work proposes a new semi-supervised loss combining an end-to-end differentiable ASR!TTS loss with TTS!ASR loss. The method is able to leverage both unpaired speech and text data to outperform recently proposed related techniques in terms of %WER. We provide extensive results analyzing the impact of data quantity and speech and text modalities and show consistent gains across WSJ and Librispeech corpora. Our code is provided in ESPnet to reproduce the experiments.

Published
2019
Pages
3790-3794
Journal
Proceedings of Interspeech - on-line, vol. 2019, no. 9, ISSN 1990-9772
Proceedings
Proceedings of Interspeech
Conference
Interspeech Conference, Graz, AT
Publisher
International Speech Communication Association
Place
Graz, AT
DOI
10.21437/Interspeech.2019-3167
UT WoS
000831796403198
EID Scopus
2-s2.0-85074698215
BibTeX 
@INPROCEEDINGS{FITPUB12089,
   author = "K. Murali Baskar and Shinji Watanabe and Ramon Astudillo and Takaaki Hori and Luk\'{a}\v{s} Burget and Jan \v{C}ernock\'{y}",
   title = "Semi-supervised Sequence-to-sequence ASR using Unpaired Speech and Text",
   pages = "3790--3794",
   booktitle = "Proceedings of Interspeech",
   journal = "Proceedings of Interspeech - on-line",
   volume = 2019,
   number = 9,
   year = 2019,
   location = "Graz, AT",
   publisher = "International Speech Communication Association",
   ISSN = "1990-9772",
   doi = "10.21437/Interspeech.2019-3167",
   language = "english",
   url = "https://www.fit.vut.cz/research/publication/12089"
}
Back to top