Mixture of Gaussian Process Experts for Predicting Sung Melodic Contour with Expressive Dynamic Fluctuations

Abstract

We present a generative model for predicting the sung melodic contour, i.e., F 0 contour, with expressive dynamic fluctuations, such as vibrato and portamento, for a given musical score. Although several studies have attempted to characterize such fluctuations, no systematic method has been developed for generating the F 0 contour with them in connection with musical notes. In our model, the relationship between a musical note sequence and F 0 contour is directly learned by a mixture of Gaussian process experts. This approach allows us to automatically characterize the fluctuations by utilizing the kernel function for each Gaussian process expert and predict the F 0 contour for an arbitrary musical note sequence. Experimental results show that our model can better predict the F 0 contour than a baseline method can. Additionally, we discuss the effective musical contexts and the amount of training data for the prediction.

Publication
In International Conference on Acoustics, Speech and Signal Processing