CUED Publications database

Data-efficient reinforcement learning in continuous state-action Gaussian-POMDPs

McAllister, RT and Rasmussen, CE (2017) Data-efficient reinforcement learning in continuous state-action Gaussian-POMDPs. In: Neural Information Processing Systems, 2017-12- to -- pp. 2041-2050..

Full text not available from this repository.


We present a data-efficient reinforcement learning method for continuous state-action systems under significant observation noise. Data-efficient solutions under small noise exist, such as PILCO which learns the cartpole swing-up task in 30s. PILCO evaluates policies by planning state-trajectories using a dynamics model. However, PILCO applies policies to the observed state, therefore planning in observation space. We extend PILCO with filtering to instead plan in belief space, consistent with partially observable Markov decisions process (POMDP) planning. This enables data-efficient learning under significant observation noise, outperforming more naive methods such as post-hoc application of a filter to policies optimised by the original (unfiltered) PILCO algorithm. We test our method on the cartpole swing-up task, which involves nonlinear dynamics and requires nonlinear control.

Item Type: Conference or Workshop Item (UNSPECIFIED)
Divisions: Div F > Computational and Biological Learning
Depositing User: Cron Job
Date Deposited: 08 Dec 2017 20:20
Last Modified: 15 Apr 2021 05:58