Probabilistic embeddings for speaker diarization

Speaker embeddings (x-vectors) extracted from very short segmentsof speech have recently been shown to give competitiveperformance in speaker diarization. We generalize thisrecipe by extracting from each speech segment, in parallel withthe x-vector, also a diagonal precision matrix, thus providinga path for the propagation of information about the quality ofthe speech segment into a PLDA scoring backend. These precisionsquantify the uncertainty about what the values of theembeddings might have been if they had been extracted fromhigh quality speech segments. The proposed probabilistic embeddings(x-vectors with precisions) are interfaced with thePLDA model by treating the x-vectors as hidden variables andmarginalizing them out. We apply the proposed probabilisticembeddings as input to an agglomerative hierarchical clustering(AHC) algorithm to do diarization in the DIHARD19 evaluationset. We compute the full PLDA likelihood by the book foreach clustering hypothesis that is considered by AHC. We dojoint discriminative training of the PLDA parameters and of theprobabilistic x-vector extractor. We demonstrate accuracy gainsrelative to a baseline AHC algorithm, applied to traditional xvectors(without uncertainty), and which uses averaging of binarylog-likelihood-ratios, rather than by-the-book scoring.