CUED Publications database

Grammar Variational Autoencoder

Kusner, MJ and Paige, B and Hernández-Lobato, JM (2017) Grammar Variational Autoencoder. In: 34th International Conference on Machine Learning, 2017-8-6 to 2017-8-11, Sydney, Australia pp. 1945-1954..

Full text not available from this repository.


Deep generative models have been wildly successful at learning coherent latent representations for continuous data such as video and audio. However, generative modeling of discrete data such as arithmetic expressions and molecular structures still poses significant challenges. Crucially, state-of-the-art methods often produce outputs that are not valid. We make the key observation that frequently, discrete data can be represented as a parse tree from a context-free grammar. We propose a variational autoencoder which encodes and decodes directly to and from these parse trees, ensuring the generated outputs are always valid. Surprisingly, we show that not only does our model more often generate valid outputs, it also learns a more coherent latent space in which nearby points decode to similar discrete outputs. We demonstrate the effectiveness of our learned models by showing their improved performance in Bayesian optimization for symbolic regression and molecular synthesis.

Item Type: Conference or Workshop Item (UNSPECIFIED)
Uncontrolled Keywords: stat.ML stat.ML
Divisions: Div F > Computational and Biological Learning
Depositing User: Cron Job
Date Deposited: 17 Jul 2017 20:02
Last Modified: 30 Mar 2021 07:25