Deep MR Brain Image Super-Resolution
Using Spatio-Structural Priors

DNSP

ABSTRACT

High resolution Magnetic Resonance (MR) images are desired for accurate diagnostics. In practice, image resolution is restricted by factors like hardware and processing constraints. Recently, deep learning methods have been shown to produce compelling state-of-the-art results for image enhancement/super-resolution. Paying particular attention to desired hi-resolution MR image structure, we propose a new regularized network that exploits image priors, namely a low-rank structure and a sharpness prior to enhance deep MR image super-resolution (SR). Our contributions are then incorporating these priors in an analytically tractable fashion as well as towards a novel prior guided network architecture that accomplishes the super-resolution task. This is particularly challenging for the low rank prior since the rank is not a differentiable function of the image matrix (and hence the network parameters), an issue we address by pursuing differentiable approximations of the rank. Sharpness is emphasized by the variance of the Laplacian which we show can be implemented by a fixed feedback layer at the output of the network. As a key extension, we modify the fixed feedback (Laplacian) layer by learning a new set of training data driven filters that are optimized for enhanced sharpness. Experiments performed on publicly available MR brain image databases and comparisons against existing state-of-the-art methods show that the proposed prior guided network offers significant practical gains in terms of improved SNR/image quality measures. Because our priors are on output images, the proposed method is versatile and can be combined with a wide variety of existing network architectures to further enhance their performance.

Low Rank Prior

Sharpness Prior (Variance of Laplacian)

visual response

Visual response of learned sharpness enhancing filters

Network Structure

SR Results

Comparison of DNSP with other competing methods for medical Image SR.

Results for Limited Training

Comparison of DNSP with other methods in a limited training setup.

Related Publications

  1. V. Cherukuri, T. Guo, S. J. Schiff and V. Monga, "Deep MR Image Super-Resolution Using Structural Priors," 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, 2018, pp. 410-414. (IEEE Xplore)
  2. V. Cherukuri, T. Guo, S. J. Schiff and V. Monga, "Deep MR Image Super-Resolution Using Spatio-Structural Priors," Accepted to IEEE Transactions on Image Processing, September 2019. (arXiv)(IEEE Xplore)

Selected References

  1. C. Dong, C. C. Loy, K. He, and X. Tang, “Image superresolution using deep convolutional networks,” IEEE transactions on pattern analysis and machine intelligence, vol. 38, no. 2, pp. 295–307, 2016.

  2. F. Shi, J. Cheng, L.Wang, P.-T. Yap, and D. Shen, “LRTV: MR image super-resolution with low-rank and total variation regularizations,” IEEE transactions on medical imaging, vol. 34, no. 12, pp. 2459–2466, 2015.

  3. D.-H. Trinh, M. Luong, F. Dibos, J.-M. Rocchisani, C.-D. Pham, and T. Q. Nguyen, “Novel example-based method for super-resolution and denoising of medical images,” IEEE Trans. on Image Processing, vol. 23, pp. 1882–1895, 2014.

Email
ipal.psu@gmail.com

Address
104 Electrical Engineering East,
University Park, PA 16802, USA

Lab Phone:
814-863-7810
814-867-4564