Deep Image Super Resolution
via Natural Image Priors

DNIP


ABSTRACT

Single image super-resolution (SR) via deep learning has recently gained significant attention in the literature. Convolutional neural networks (CNNs) are typically learned to represent the mapping between lo-res (LR) and hi-res (HR) images/patches with the help of training examples. Most existing deep networks for SR produce high quality results when training data is abundant. However, their performance degrades sharply when training is limited. We propose to regularize deep structures with prior knowledge about the images so that they can capture more structural information from the same limited data. In particular, we incorporate in a tractable fashion within the CNN framework, natural image priors which have shown to have much recent success in imaging and vision inverse problems. Experimental results show that the proposed deep network with natural image priors is particularly effective in training starved regimes.

Network Structure

Evaluations

PSNR values with different amount (%) of training data for DNIP and VDSR with various number of layers.

SR Results

Testing image ppt3.bmp from dataset Set 14. The assessments are displayed under the SR results from different methods as (PSNR, SSIM).

Related Publications

  1. H. S.Mousavi, T. Guo, and V. Monga, “Deep Image Super Resolution via Natural Image Priors,” accepted to the IEEE International Conference on Acoustics, Speech, and Signal Processing, April 2018.[IEEE Xplore]

Selected References

  1. J. Kim, J. K. Lee, and K. M. Lee, “Accurate image super-resolution using very deep convolutional networks,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR Oral), June 2016 [paper].

  2. D. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014. [paper].

  3. C. Dong, C. C. Loy, K. He, and X. Tang, “Learning a deep convolutional network for image super-resolution,” in Computer Vision–ECCV 2014, pp. 184–199, Springer, 2014. [paper].

  4. C. Dong, C. C. Loy, and X. Tang, “Accelerating the super-resolution convolutional neural network,” in European Conference on Computer Vision, pp. 391–407, Springer, 2016.[ paper].

  5. J.-B. Huang, A. Singh, and N. Ahuja, “Single image superresolution from transformed self-exemplars,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5197–5206, 2015.[paper].

  6. Z. Wang, D. Liu, J. Yang, W. Han, and T. Huang, “Deep networks for image super-resolution with sparse prior,” in Proceedings of the IEEE International Conference on Computer Vision, pp. 370–378, 2015. [paper].

  7. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778, 2016.[paper].

Email
ipal.psu@gmail.com

Address
104 Electrical Engineering East,
University Park, PA 16802, USA

Lab Phone:
814-863-7810
814-867-4564