CUED Publications database

Icebreaker: Element-wise efficient information acquisition with a Bayesian deep latent gaussian model

Gong, W and Tschiatschek, S and Turner, RE and Nowozin, S and Hernández-Lobato, JM and Zhang, C (2019) Icebreaker: Element-wise efficient information acquisition with a Bayesian deep latent gaussian model. Advances in Neural Information Processing Systems, 32. ISSN 1049-5258

Full text not available from this repository.


© 2019 Neural information processing systems foundation. All rights reserved. In this paper, we address the ice-start problem, i.e., the challenge of deploying machine learning models when only a little or no training data is initially available, and acquiring each feature element of data is associated with costs. This setting is representative of the real-world machine learning applications. For instance, in the health-care domain, obtaining every single measurement comes with a cost. We propose Icebreaker, a principled framework for element-wise training data acquisition. Icebreaker introduces a full Bayesian Deep Latent Gaussian Model (BELGAM) with a novel inference method, which combines recent advances in amortized inference and stochastic gradient MCMC to enable fast and accurate posterior inference. By utilizing BELGAM's ability to fully quantify model uncertainty, we also propose two information acquisition functions for imputation and active prediction problems. We demonstrate that BELGAM performs significantly better than previous variational autoencoder (VAE) based models, when the data set size is small, using both machine learning benchmarks and real-world recommender systems and health-care applications. Moreover, Icebreaker not only demonstrates improved performance compared to baselines, but it is also capable of achieving better test performance with less training data available.

Item Type: Article
Uncontrolled Keywords: cs.LG cs.LG cs.AI stat.ML
Divisions: Div F > Computational and Biological Learning
Depositing User: Cron Job
Date Deposited: 01 Oct 2019 03:38
Last Modified: 17 Sep 2020 02:57