CUED Publications database

Integrated speaker-adaptive speech synthesis

Wan, M and Degottex, G and Gales, MJF and IEEE, (2017) Integrated speaker-adaptive speech synthesis. In: 2017 IEEE AUTOMATIC SPEECH RECOGNITION AND UNDERSTANDING WORKSHOP (ASRU), 2017-12-16 to 2017-12-20, Okinawa, Japan pp. 705-711..

Full text not available from this repository.


Enabling speech synthesis systems to rapidly adapt to sound like a particular speaker is an essential attribute for building personalised systems. For deep-learning based approaches, this is difficult as these networks use a highly distributed representation. It is not simple to interpret the model parameters, which complicates the adaptation process. To address this problem, speaker characteristics can be encapsulated in fixed-length speaker-specific Identity Vectors (iVectors), which are appended to the input of the synthesis network. Altering the iVector changes the nature of the synthesised speech. The challenge is to derive an optimal iVector for each speaker that encodes all the speaker attributes required for the synthesis system. The standard approach involves two separate stages: estimation of the iVectors for the training data; and training the synthesis network. This paper proposes an integrated training scheme for speaker adaptive speech synthesis. For the iVector extraction, an attention based mechanism, which is a function of the context labels, is used to combine the data from the target speaker. This attention mechanism, as well as nature of the features being merged, are optimised at the same time as the synthesis network parameters. This should yield an iVector-like speaker representation that is optimal for use with the synthesis system. The system is evaluated on the Voice Bank corpus. The resulting system automatically provides a sensible attention sequence and shows improved performance from the standard approach.

Item Type: Conference or Workshop Item (UNSPECIFIED)
Uncontrolled Keywords: speech synthesis iVector integrated adaptation attention mechanism
Divisions: Div F > Machine Intelligence
Depositing User: Cron Job
Date Deposited: 26 Apr 2018 01:29
Last Modified: 15 Apr 2021 06:01
DOI: 10.1109/ASRU.2017.8269006