Geometric loss functions for camera pose regression with deep learning

Kendall, A and Cipolla, R and IEEE, (2017) Geometric loss functions for camera pose regression with deep learning. In: 30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017-7-22 to -- pp. 6555-6564..

Abstract

Deep learning has shown to be effective for robust and real-time monocular image relocalisation. In particular, PoseNet [22] is a deep convolutional neural network which learns to regress the 6-DOF camera pose from a single im- age. It learns to localize using high level features and is ro- bust to difficult lighting, motion blur and unknown camera intrinsics, where point based SIFT registration fails. How- ever, it is trained using a naive loss function, with hyper- parameters which require expensive tuning. In this paper, we give the problem a more fundamental theoretical treat- ment. We explore a number of novel loss functions for learning camera pose which are based on geometry and scene reprojection error. Additionally we show how to au- tomatically learn an optimal weighting to simultaneously regress position and orientation. By leveraging geometry, we demonstrate that our technique significantly improves PoseNet’s performance across datasets ranging from indoor rooms to a small city.