Information Processing & Algorithm Laboratory

In this paper we study the problem of image alignment for panoramic stitching. Unlike most existing approaches that are feature-based, our algorithm works on pixels directly, and accounts for errors across the whole images globally. Technically, we formulate the alignment problem as rank-1 and sparse matrix decomposition over transformed images, and develop an efficient algorithm for solving this challenging non-convex optimization problem. The algorithm reduces to a sequence of subproblems, where we analytically establish exact recovery conditions, convergence and optimality, together with convergence rate and complexity. We generalize it to simultaneously align multiple images and recover multiple homographies, extending its application scope towards vast majority of practical scenarios. Experimental results demonstrate that the proposed algorithm is capable of more accurately aligning the images and generating higher quality stitched images than state-of-the-art methods.

Click here to download the source code for our prosed BRAS algorithm (BRAS).

Click here to download the catabus image set

Clicke here to download high resolution results for all algorithms

apartments dataset (hover and scroll your mouse to enlarge, desktop only)[5]

catabus dataset (hover and scroll your mouse to enlarge, desktop only)

railtracks dataset (hover and scroll your mouse to enlarge, desktop only)[4]

Y. Li and V. Monga, SIASM: Sparsity-based Image Alignment and Stitching Method for Robust Image Mosaicking, IEEE International Conference on Image Processing, Phoenix, Sep 25th-28th, 2016. [IEEE Xplore]
Y. Li, M. Tofighi and V. Monga, "Robust Alignment for Panoramic Stitching Via an Exact Rank Constraint," IEEE Transactions on Image Processing, vol. 28, no. 10, pp. 4730-4745, Oct. 2019. [local PDF][IEEE Xplore]

M. Brown and D. G. Lowe, Automatic Panoramic Image Stitching using Invariant Features, Int'l J. Comput. Vis., vol. 60, no. 2, pp. 91-110, Nov. 2004.
J. Hu, D. Q. Zhang, H. Yu and C. W. Chen, Multi-objective content preserving warping for image stitching, in Proc. IEEE ICME, 2015.
C. H. Chang, Y. Sato and Y. Y. Chuang, Shape-Preserving Half-Projective Warps for Image Stitching, in Proc. IEEE Conf. CVPR, Jun. 2014.
J. Zaragoza, T. J. Chin, Q. H. Tran, M. S. Brown and D. Suter, As-Projective-As-Possible Image Stitching with Moving DLT, IEEE Trans. Pattern Anal. Mach. Intelli., vol. 36, no. 7, pp. 1285-1298, Jul. 2014.
J. Gao, S. J. Kim and M. Brown, Constructing image panoramas using dual-homography warping, in Proc. IEEE Conf. CVPR, Jun. 2011.
W. Y. Lin, S. Liu, Y. Matsushita, T. T. Ng and L. F. Cheong, Smoothly varying affine stitching, in Proc. IEEE Conf. CVPR, Jun. 2011.
Y. Peng, A. Ganesh, J. Wright, W. Xu and Y. Ma, RASL: Robust alignment by sparse and low-rank decomposition for linearly correlated images,IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 11, pp. 2233-2246, Nov. 2012.

Robust Alignment for Panoramic Stitching
via an Exact Rank Constraint

BRAS

ABSTRACT

Source Code and Image Set

Stitched Images

Alignment without Stitching

Related Publications

Selected References

Robust Alignment for Panoramic Stitchingvia an Exact Rank Constraint