Fast and Robust Techniques for 3D/2D Registration and Blending of Photographs on Massive Point Clouds

We present a simple, fast and robust technique for semi-automatic 2D-3D registration capable to align a large set of unordered images to a massive point cloud with minimal human effort. Our method converts the hard to solve image-to-geometry registration problem in a Structure-from-Motion (SfM) plus a 3D-3D registration problem. We exploit a SfM framework that, starting just from the unordered image collection, computes an estimate of camera parameters and a sparse 3D geometry deriving from matched image features. We then coarsely register this model to the given 3D geometry by estimating a global scale and absolute orientation using minimal manual intervention. A specialized sparse bundle adjustment (SBA) step, exploiting the correspondence between the model deriving from image features and the fine input 3D geometry, is then used to refine intrinsic and extrinsic parameters of each camera. Output data is suitable for photo blending frameworks to produce seamless colored models. In this sense, we also present an efficient scalable streaming technique for mapping highly detailed color information on extremely dense point clouds. It does not require meshing or extensive processing of the input model, works on a coarsely spatially-reordered point stream and can adaptively refine point cloud geometry on the basis of image content. Seamless multi-band image blending is obtained by using GPU accelerated screenspace operators, which solve point set visibility, compute a per-pixel view-dependent weight and ensure a smooth weighting function over each input image. The proposed approach works independently on each image in a memory coherent manner, and can be easily extended to include further image quality estimators. The effectiveness of the method is demonstrated on a series of real-world Cultural Heritage datasets.

@InProceedings{Pin12,
  author       = {Pintus, R.},
  title        = {Fast and Robust Techniques for 3D/2D Registration and Blending of Photographs on Massive Point Clouds},
  booktitle    = {Collana seminari interni 2012},
  number       = {20120314},
  month        = {march},
  year         = {2012},
  keywords     = {semi-automatic 2D-3D registration, alignment, unordered images, massive point cloud, Cultural Heritage datasets},
  url          = {https://publications.crs4.it/pubdocs/2012/Pin12},
}