Deep clustering-based beamforming for separation with unknown number of sources

source separation, source counting, timefrequency masking, beamforming

This article is about the deep clustering-based beamforming for separation with unknown number of sources.

This paper extends a deep clustering algorithm for use with
time-frequency masking-based beamforming and perform separation
with an unknown number of sources. Deep clustering
is a recently proposed single-channel source separation algorithm,
which projects inputs into the embedding space and
performs clustering in the embedding domain. In deep clustering,
bi-directional long short-term memory (BLSTM) recurrent
neural networks are trained to make embedding vectors
orthogonal for different speakers and concurrent for the same
speaker. Then, by clustering the embedding vectors at test time,
we can estimate time-frequency masks for separation. In this
paper, we extend the deep clustering algorithm to a multiple
microphone setup and incorporate deep clustering-based timefrequency
mask estimation into masking-based beamforming,
which has been shown to be more effective than masking for
automatic speech recognition. Moreover, we perform source
counting by computing the rank of the covariance matrix of the
embedding vectors. With our proposed approach, we can perform
masking-based beamforming in a multiple-speaker case
without knowing the number of speakers. Experimental results
show that our proposed deep clustering-based beamformer
achieves comparable source separation performance to that obtained
with a complex Gaussian mixture model-based beamformer,
which requires the number of sources in advance for
mask estimation.