2D-3D registration of deformable shapes with manifold projection

Abstract

We present an algorithm able to register a known 3D deformable model to a set of 2D matched points extracted from a single image. Unlike previous approaches, the problem is solved simultaneously for both the rigid and non-rigid parameters of the model. The key advantage of our approach is the projection of an initial affine estimation of the motion parameters into the motion manifold corresponding to the exact parametrization of the problem. This projection is formulated as theminimization of the distance between the affine solution and the surface of the manifold. Such optimization results in a quadratically constrained quadratic minimization problem that can be efficiently solved with standard optimization tools. Synthetic and real tests demonstrate the effectiveness of the approach.