A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Advanced Search
Volume 12 Issue 6
Jun.  2025

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 18.3, Top 1 (SCI Q1)
    CiteScore: 28.2, Top 1% (Q1)
    Google Scholar h5-index: 95， TOP 5
Turn off MathJax
Article Contents
Article Navigation >  IEEE/CAA Journal of Automatica Sinica  > 2025  >  12(6): 1159-1175
Y. Du, Y. Ma, J. Huang, X. Mei, J. Qin, and F. Fan, “Joint super-resolution and nonuniformity correction model for infrared light field images based on frequency correlation learning,” IEEE/CAA J. Autom. Sinica, vol. 12, no. 6, pp. 1159–1175, Jun. 2025. doi: 10.1109/JAS.2024.124881
Citation: Y. Du, Y. Ma, J. Huang, X. Mei, J. Qin, and F. Fan, “Joint super-resolution and nonuniformity correction model for infrared light field images based on frequency correlation learning,” IEEE/CAA J. Autom. Sinica, vol. 12, no. 6, pp. 1159–1175, Jun. 2025. doi: 10.1109/JAS.2024.124881
shu

Joint Super-Resolution and Nonuniformity Correction Model for Infrared Light Field Images Based on Frequency Correlation Learning

doi: 10.1109/JAS.2024.124881
Funds:  This work was supported by the National Natural Science Foundation of China (62475199, 62075169, U23B2050) and the Industry University-Research Cooperation Program of Zhuhai (2220004002828)
More Information
    Abstract
  • Super-resolution (SR) for the camera array-based infrared light field (IRLF) images aims to reconstruct high-resolution sub-aperture images (SAIs) from their low-resolution counterparts. Existing SR methods mainly focus on exploiting the spatial and angular information of SAIs and have achieved promising results in the visible band. However, they fail to adaptively correct the nonuniform noise in IRLF images, resulting in over-smoothness or artifacts in their results. This study proposes a novel method that reconstructs high-resolution IRLF images while correcting the nonuniformity. The main idea is to decompose the structure and nonuniform noise into high- and low-frequency components and then learn the frequency correlations to help correct the nonuniformity. To learn the frequency correlation, intra- and inter-frequency units are designed. The former learns the correlation of neighboring pixels within each component, aiming to reconstruct the structure and coarsely remove nonuniform noise. The latter models the correlation of contents between different components to reconstruct fine-grained structures and reduce residual noise. Both units are equipped with our designed triple-attention mechanism, which can jointly exploit spatial, angular, and frequency information. Moreover, we collected two real-world IRLF-image datasets with significant nonuniformity, which can be used as a common base in the field. Qualitative and quantitative comparisons demonstrate that our method outperforms state-of-the-art approaches with a clearer structure and fewer artifacts. The code is available at https://github.com/DuYou2023/IRLF-FSR.

     

    • FullText(HTML)
  • loading
    • References(62)
  • [1]
    Y. Ma, X. Wang, W. Gao, Y. Du, J. Huang, and F. Fan, “Progressive fusion network based on infrared light field equipment for infrared image enhancement,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 9, pp. 1687–1690, 2022. doi: 10.1109/JAS.2022.105812
    [2]
    G. Liu, H. Yue, J. Wu, and J. Yang, “Intra-inter view interaction network for light field image super-resolution,” IEEE Trans. Multimedia, vol. 25, pp. 256–266, 2021.
    [3]
    Y. Wang, L. Wang, J. Yang, W. An, J. Yu, and Y. Guo, “Spatial-angular interaction for light field image super-resolution,” in Proc. European Conf. Computer Vision. Springer, 2020, pp. 290–308.
    [4]
    Y. Ding, Z. Chen, Y. Ji, J. Yu, and J. Ye, “Light field-based underwater 3d reconstruction via angular re-sampling,” IEEE Trans. Computational Imaging, vol. 9, pp. 881–893, 2023. doi: 10.1109/TCI.2023.3319983
    [5]
    Z. Cai, X. Liu, G. Pedrini, W. Osten, and X. Peng, “Structured-light-field 3d imaging without phase unwrapping,” Optics and Lasers in Engineering, vol. 129, p. 106047, 2020. doi: 10.1016/j.optlaseng.2020.106047
    [6]
    J. Hur, J. Y. Lee, J. Choi, and J. Kim, “I see-through you: A framework for removing foreground occlusion in both sparse and dense light field images,” in Proc. the IEEE/CVF Winter Conf. Applications of Computer Vision, 2023, pp. 229–238.
    [7]
    X. Wang, J. Liu, S. Chen, and G. Wei, “Effective light field de-occlusion network based on swin transformer,” IEEE Trans. Circuits and Systems for Video Technology, vol. 33, no. 6, pp. 2590–2599, 2023. doi: 10.1109/TCSVT.2022.3226227
    [8]
    W. Yan, X. Zhang, and H. Chen, “Occlusion-aware unsupervised light field depth estimation based on muti-scale gans,” IEEE Trans. Circuits and Systems for Video Technology, pp. 1–1, 2024.
    [9]
    Z. Cui, H. Sheng, D. Yang, S. Wang, R. Chen, and W. Ke, “Light field depth estimation for non-lambertian objects via adaptive cross operator,” IEEE Trans. Circuits and Systems for Video Technology, vol. 34, no. 2, pp. 1199–1211, 2024. doi: 10.1109/TCSVT.2023.3292884
    [10]
    A. Glowacz, “Ventilation diagnosis of minigrinders using thermal images,” Expert Systems with Applications, vol. 237, p. 121435, 2024. doi: 10.1016/j.eswa.2023.121435
    [11]
    A. Glowacz, “Thermographic fault diagnosis of electrical faults of commutator and induction motors,” Engineering Applications of Artificial Intelligence, vol. 121, p. 105962, 2023. doi: 10.1016/j.engappai.2023.105962
    [12]
    K. Prajapati, V. Chudasama, H. Patel, A. Sarvaiya, K. P. Upla, K. Raja, R. Ramachandra, and C. Busch, “Channel split convolutional neural network (chasnet) for thermal image super-resolution,” in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition, 2021, pp. 4368–4377.
    [13]
    Y. Huang, Z. Jiang, R. Lan, S. Zhang, and K. Pi, “Infrared image super-resolution via transfer learning and psrgan,” IEEE Signal Processing Letters, vol. 28, pp. 982–986, 2021. doi: 10.1109/LSP.2021.3077801
    [14]
    L. Sun, Z. Liu, X. Sun, L. Liu, R. Lan, and X. Luo, “Lightweight image super-resolution via weighted multi-scale residual network,” IEEE/CAA J. Autom. Sinica, vol. 8, no. 7, pp. 1271–1280, 2021. doi: 10.1109/JAS.2021.1004009
    [15]
    Y. Huang, T. Miyazaki, X. Liu, and S. Omachi, “Infrared image super-resolution: Systematic review, and future trends,” arXiv preprint arXiv: 2212.12322, 2022.
    [16]
    Y. Wang, L. Wang, G. Wu, J. Yang, W. An, J. Yu, and Y. Guo, “Disentangling light fields for super-resolution and disparity estimation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 45, no. 1, pp. 425–443, 2022.
    [17]
    Z. Liang, Y. Wang, L. Wang, J. Yang, S. Zhou, and Y. Guo, “Learning non-local spatial-angular correlation for light field image super-resolution,” in Proc. IEEE/CVF Int. Conf. Computer Vision, 2023, pp. 12376–12386.
    [18]
    R. A. Farrugia, C. Galea, and C. Guillemot, “Super resolution of light field images using linear subspace projection of patch-volumes,” IEEE J. Selected Topics in Signal Processing, vol. 11, no. 7, pp. 1058–1071, 2017. doi: 10.1109/JSTSP.2017.2747127
    [19]
    M. Rossi and P. Frossard, “Geometry-consistent light field super-resolution via graph-based regularization,” IEEE Trans. Image Processing, vol. 27, no. 9, pp. 4207–4218, 2018. doi: 10.1109/TIP.2018.2828983
    [20]
    V. K. Ghassab and N. Bouguila, “Light field super-resolution using edge-preserved graph-based regularization,” IEEE Trans. Multimedia, vol. 22, no. 6, pp. 1447–1457, 2019.
    [21]
    K. Jin, A. Yang, Z. Wei, S. Guo, M. Gao, and X. Zhou, “Distgepit: Enhanced disparity learning for light field image super-resolution,” in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition, 2023, pp. 1373–1383.
    [22]
    R. Cong, H. Sheng, D. Yang, Z. Cui, and R. Chen, “Exploiting spatial and angular correlations with deep efficient transformers for light field image super-resolution,” IEEE Trans. Multimedia,