Globally optimal solution to exploit rigidity when recovering structure from motion under occlusion

Abstract

Widely used SVD-based matrix factorization approaches to the recovery of 3D rigid structure from motion (SFM), require a set of feature points to be visible in a set of images. When there is occlusion, several feature points disappear, the observation matrix misses some entries, there is not equivalent to the SVD, and only suboptimal solutions have been proposed to exploit rigidity. In this paper, we propose a method to complete the trajectories that correspond to a rigid scene, in an optimal way. Our algorithm is not iterative (thus avoiding problems like sensitivity to initialization and local optima); it rather computes in a finite number of steps the globally optimal completion of the observation matrix. We describe experiments that illustrate the gain in accuracy of SFM.