CUED Publications database

Rényi Divergence Variational Inference

Li, Y and Turner, R (2016) Rényi Divergence Variational Inference. In: Neural Information Processing Systems (NIPS 2016), 2016-12-5 to 2016-12-10, Barcelona, Spain pp. 1073-1081..

Full text not available from this repository.


This paper introduces the $\textit{variational Rényi bound}$ (VR) that extends traditional variational inference to Rényi’s $\alpha$-divergences. This new family of variational methods unifies a number of existing approaches, and enables a smooth interpolation from the evidence lower-bound to the log (marginal) likelihood that is controlled by the value of $\alpha$ that parametrises the divergence. The reparameterization trick, Monte Carlo approximation and stochastic optimisation methods are deployed to obtain a tractable and unified framework for optimisation. We further consider negative $\alpha$ values and propose a novel variational inference method as a new special case in the proposed framework. Experiments on Bayesian neural networks and variational auto-encoders demonstrate the wide applicability of the VR bound.

Item Type: Conference or Workshop Item (UNSPECIFIED)
Divisions: Div F > Computational and Biological Learning
Depositing User: Cron Job
Date Deposited: 17 Jul 2017 20:01
Last Modified: 22 May 2018 06:55