CUED Publications database

Expectation propagation in linear regression models with spike-and-slab priors

Hernández-Lobato, JM and Hernández-Lobato, D and Suárez, A (2015) Expectation propagation in linear regression models with spike-and-slab priors. Machine Learning, 99. pp. 437-487. ISSN 0885-6125

Full text not available from this repository.


© 2014, The Author(s). An expectation propagation (EP) algorithm is proposed for approximate inference in linear regression models with spike-and-slab priors. This EP method is applied to regression tasks in which the number of training instances is small and the number of dimensions of the feature space is large. The problems analyzed include the reconstruction of genetic networks, the recovery of sparse signals, the prediction of user sentiment from customer-written reviews and the analysis of biscuit dough constituents from NIR spectra. The proposed EP method outperforms in most of these tasks another EP method that ignores correlations in the posterior and a variational Bayes technique for approximate inference. Additionally, the solutions generated by EP are very close to those given by Gibbs sampling, which can be taken as the gold standard but can be much more computationally expensive. In the tasks analyzed, spike-and-slab priors generally outperform other sparsifying priors, such as Laplace, Student’s $$t$$t and horseshoe priors. The key to the improved predictions with respect to Laplace and Student’s $$t$$t priors is the superior selective shrinkage capacity of the spike-and-slab prior distribution.

Item Type: Article
Divisions: Div F > Computational and Biological Learning
Depositing User: Cron Job
Date Deposited: 17 Jul 2017 19:29
Last Modified: 17 Aug 2017 01:24