Publication Details

Parallel Training of Neural Networks for Speech Recognition

VESELÝ Karel, BURGET Lukáš and GRÉZL František. Parallel Training of Neural Networks for Speech Recognition. In: Proceedings of the 11th Annual Conference of the International Speech Communication Association (INTERSPEECH 2010). Makuhari, Chiba: International Speech Communication Association, 2010, pp. 2934-2937. ISSN 1990-9772.
Czech title
Paralelní trénování neuronových sítí pro rozpoznávání řeči
Type
conference paper
Language
english
Authors
Veselý Karel, Ing. (FIT BUT)
Burget Lukáš, doc. Ing., Ph.D. (DCGM FIT BUT)
Grézl František, Ing., Ph.D. (DCGM FIT BUT)
URL
Keywords

Artificial Neural Network, GPU, CUDA, Phoneme Classification, Fast Training

Abstract

The paper is on study and implementation of two different approaches to the parallelization of the ANN training procedure for sequential patterns.

Annotation

In this paper we describe parallel implementation of ANN training procedure based on block mode back-propagation learning algorithm. Two different approaches to parallelization were implemented. The first is data parallelization using POSIX threads, it is suitable for multi-core computers. The second is node parallelization using high performance SIMD architecture of GPU with CUDA, suitable for CUDA enabled computers. We compare the speed-up of both approaches by learning typically-sized network on the real-world phoneme-state classification task, showing nearly 10 times reduction when using CUDA version, while the 8-core server with multi-thread version gives only 4 times reduction. In both cases we compared to an already BLAS optimized implementation. The training tool will be released as Open-Source software under project name TNet.

Published
2010
Pages
2934-2937
Journal
Proceedings of Interspeech - on-line, vol. 2010, no. 9, ISSN 1990-9772
Proceedings
Proceedings of the 11th Annual Conference of the International Speech Communication Association (INTERSPEECH 2010)
Conference
Interspeech Conference, Tokyo, JP
Publisher
International Speech Communication Association
Place
Makuhari, Chiba, JP
BibTeX 
@INPROCEEDINGS{FITPUB9364,
   author = "Karel Vesel\'{y} and Luk\'{a}\v{s} Burget and Franti\v{s}ek Gr\'{e}zl",
   title = "Parallel Training of Neural Networks for Speech Recognition",
   pages = "2934--2937",
   booktitle = "Proceedings of the 11th Annual Conference of the International Speech Communication Association (INTERSPEECH 2010)",
   journal = "Proceedings of Interspeech - on-line",
   volume = 2010,
   number = 9,
   year = 2010,
   location = "Makuhari, Chiba, JP",
   publisher = "International Speech Communication Association",
   ISSN = "1990-9772",
   language = "english",
   url = "https://www.fit.vut.cz/research/publication/9364"
}
Back to top