A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Volume 9 Issue 8
Aug.  2022

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 11.8, Top 4% (SCI Q1)
    CiteScore: 17.6, Top 3% (Q1)
    Google Scholar h5-index: 77, TOP 5
Turn off MathJax
Article Contents
R. B. Jin, M. Wu, K. Y. Wu, K. Z. Gao, Z. H. Chen, and X. L. Li, “Position encoding based convolutional neural networks for machine remaining useful life prediction,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 8, pp. 1427–1439, Aug. 2022. doi: 10.1109/JAS.2022.105746
Citation: R. B. Jin, M. Wu, K. Y. Wu, K. Z. Gao, Z. H. Chen, and X. L. Li, “Position encoding based convolutional neural networks for machine remaining useful life prediction,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 8, pp. 1427–1439, Aug. 2022. doi: 10.1109/JAS.2022.105746

Position Encoding Based Convolutional Neural Networks for Machine Remaining Useful Life Prediction

doi: 10.1109/JAS.2022.105746
Funds:  This work was supported by National Research Foundation of Singapore, AME Young Individual Research Grant (A2084c0167)
More Information
  • Accurate remaining useful life (RUL) prediction is important in industrial systems. It prevents machines from working under failure conditions, and ensures that the industrial system works reliably and efficiently. Recently, many deep learning based methods have been proposed to predict RUL. Among these methods, recurrent neural network (RNN) based approaches show a strong capability of capturing sequential information. This allows RNN based methods to perform better than convolutional neural network (CNN) based approaches on the RUL prediction task. In this paper, we question this common paradigm and argue that existing CNN based approaches are not designed according to the classic principles of CNN, which reduces their performances. Additionally, the capacity of capturing sequential information is highly affected by the receptive field of CNN, which is neglected by existing CNN based methods. To solve these problems, we propose a series of new CNNs, which show competitive results to RNN based methods. Compared with RNN, CNN processes the input signals in parallel so that the temporal sequence is not easily determined. To alleviate this issue, a position encoding scheme is developed to enhance the sequential information encoded by a CNN. Hence, our proposed position encoding based CNN called PE-Net is further improved and even performs better than RNN based methods. Extensive experiments are conducted on the C-MAPSS dataset, where our PE-Net shows state-of-the-art performance.

     

  • loading
  • [1]
    Z. H. Chen, M. Wu, R. Zhao, F. Guretno, R. Q. Yan, and X. L. Li, “Machine remaining useful life prediction via an attention-based deep learning approach,” IEEE Trans. Ind. Electron., vol. 68, no. 3, pp. 2521–2531, Mar. 2021. doi: 10.1109/TIE.2020.2972443
    [2]
    K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in Proc. 3rd Int. Conf. Learning Representations, San Diego, USA, 2015.
    [3]
    K. M. He, X. Y. Zhang, S. Q. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Las Vegas, USA, 2016, pp. 770–778.
    [4]
    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Proc. 31st Int. Conf. Neural Information Processing Systems, Long Beach, USA, 2017, pp. 6000–6010.
    [5]
    J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proc. Conf. North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, Minnesota, 2018, pp. 4171–4186.
    [6]
    A. T. Khan, S. Li, and X. W. Cao, “Human guided cooperative robotic agents in smart home using beetle antennae search,” Sci. China Inf. Sci., vol. 65, no. 2, Article No. 122204, Jan. 2022. doi: 10.1007/s11432-020-3073-5
    [7]
    A. T. Khan, X. W. Cao, Z. Li, and S. Li, “Enhanced beetle antennae search with zeroing neural network for online solution of constrained optimization,” Neurocomputing, vol. 447, pp. 294–306, Aug. 2021. doi: 10.1016/j.neucom.2021.03.027
    [8]
    A. T. Khan, S. Li, and Z. Li, “Obstacle avoidance and model-free tracking control for home automation using bio-inspired approach,” Adv. Control Appl., vol. 4, p. e63, Dec. 2021.
    [9]
    J. D. Lin, Z. Lin, G. B. Liao, and H. P. Yin, “A novel product remaining useful life prediction approach considering fault effects,” IEEE/CAA J. Autom. Sinica, vol. 8, no. 11, pp. 1762–1773, Nov. 2021. doi: 10.1109/JAS.2021.1004168
    [10]
    Q. Xu, Z. H. Chen, K. Y. Wu, C. Wang, M. Wu, and X. L. Li, “KDnet-RUL: A knowledge distillation framework to compress deep neural networks for machine remaining useful life prediction,” IEEE Trans. Ind. Electron., vol. 69, no. 2, pp. 2022–2032, Feb. 2022. doi: 10.1109/TIE.2021.3057030
    [11]
    Y. Qin, D. L. Chen, S. Xiang, and C. C. Zhu, “Gated dual attention unit neural networks for remaining useful life prediction of rolling bearings,” IEEE Trans. Ind. Inf., vol. 17, no. 9, pp. 6438–6447, Sep. 2021. doi: 10.1109/TII.2020.2999442
    [12]
    X. W. Zhang, Y. Qin, C. Yuen, L. Jayasinghe, and X. Liu, “Time-series regeneration with convolutional recurrent generative adversarial network for remaining useful life estimation,” IEEE Trans. Ind. Inf., vol. 17, no. 10, pp. 6820–6831, Oct. 2021. doi: 10.1109/TII.2020.3046036
    [13]
    B. Y. Yang, R. N. Liu, and E. Zio, “Remaining useful life prediction based on a double-convolutional neural network architecture,” IEEE Trans. Ind. Electron., vol. 66, no. 12, pp. 9521–9530, Dec. 2019. doi: 10.1109/TIE.2019.2924605
    [14]
    G. S. Babu, P. L. Zhao, and X. L. Li, “Deep convolutional neural network based regression approach for estimation of remaining useful life,” in Proc. 21st Int. Conf. Database Systems for Advanced Applications, Dallas, USA, 2016, pp. 214–228.
    [15]
    S. Zheng, K. Ristovski, A. Farahat, and C. Gupta, “Long short-term memory network for remaining useful life estimation,” in Proc. IEEE Int. Conf. Prognostics and Health Management, Dallas, USA, 2017, pp. 88–95.
    [16]
    J. J. Zhang, P. Wang, R. Q. Yan, and R. X. Gao, “Long short-term memory for machine remaining life prediction,” J. Manuf. Syst., vol. 48, pp. 78–86, Jul. 2018. doi: 10.1016/j.jmsy.2018.05.011
    [17]
    C. G. Huang, H. Z. Huang, and Y. F. Li, “A bidirectional LSTM prognostics method under multiple operational conditions,” IEEE Trans. Ind. Electron., vol. 66, no. 11, pp. 8792–8802, Nov. 2019. doi: 10.1109/TIE.2019.2891463
    [18]
    L. Ren, J. B. Dong, X. K. Wang, Z. H. Meng, L. Zhao, and M. J. Deen, “A data-driven auto-CNN-LSTM prediction model for Lithium-Ion battery remaining useful life,” IEEE Trans. Ind. Inf., vol. 17, no. 5, pp. 3478–3487, May 2021. doi: 10.1109/TII.2020.3008223
    [19]
    L. Jayasinghe, T. Samarasinghe, C. Yuenv, J. C. N. Low, and S. S. Ge, “Temporal convolutional memory networks for remaining useful life estimation of industrial machinery,” in Proc. IEEE Int. Conf. Industrial Technology, Melbourne, Australia, 2019, pp. 915–920.
    [20]
    M. Xia, T. Li, T. X. Shu, J. F. Wan, C. W. De Silva, and Z. R. Wang, “A two-stage approach for the remaining useful life prediction of bearings using deep neural networks,” IEEE Trans. Ind. Inf., vol. 15, no. 6, pp. 3703–3711, Jun. 2019. doi: 10.1109/TII.2018.2868687
    [21]
    R. H. Jiao, K. X. Peng, and J. Dong, “Remaining useful life prediction for a roller in a hot strip mill based on deep recurrent neural networks,” IEEE/CAA J. Autom. Sinica, vol. 8, no. 7, pp. 1345–1354, Jul. 2021. doi: 10.1109/JAS.2021.1004051
    [22]
    C. Chen, N. Y. Lu, B. Jiang, and C. S. Wang, “A risk-averse remaining useful life estimation for predictive maintenance,” IEEE/CAA J. Autom. Sinica, vol. 8, no. 2, pp. 412–422, Feb. 2021. doi: 10.1109/JAS.2021.1003835
    [23]
    X. Li, Q. Ding, and J. Q. Sun, “Remaining useful life estimation in prognostics using deep convolution neural networks,” Reliab. Eng. Syst. Saf., vol. 172, pp. 1–11, Apr. 2018. doi: 10.1016/j.ress.2017.11.021
    [24]
    M. Ragab, Z. H. Chen, M. Wu, C. K. Kwoh, R. Q. Yan, and X. L. Li, “Attention-based sequence to sequence model for machine remaining useful life prediction,” Neurocomputing, vol. 466, pp. 58–68, Nov. 2021. doi: 10.1016/j.neucom.2021.09.022
    [25]
    J. Y. Wu, M. Wu, Z. H. Chen, X. L. Li, and R. Q. Yan, “Degradation-aware remaining useful life prediction with LSTM autoencoder,” IEEE Trans. Instrum. Meas., vol. 70, pp. 1–10, Feb. 2021.
    [26]
    C. Szegedy, W. Liu, Y. Q. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Boston, USA, 2015, pp. 1–9.
    [27]
    A. G. Howard, M. L. Zhu, B. Chen, D. Kalenichenko, W. J. Wang, T. Weyand, M. Andreetto, and H. Adam, “MobileNets: Efficient convolutional neural networks for mobile vision applications,” arXiv: 1704.04861, Apr. 2017.
    [28]
    D. Wang, F. F. Yang, K. L. Tsui, Q. Zhou, and S. J. Bae, “Remaining useful life prediction of Lithium-Ion batteries based on spherical cubature particle filter,” IEEE Trans. Instrum. Meas., vol. 65, no. 6, pp. 1282–1291, Jun. 2016. doi: 10.1109/TIM.2016.2534258
    [29]
    M. Jouin, R. Gouriveau, D. Hissel, M. C. Péra, and N. Zerhouni, “Particle filter-based prognostics: Review, discussion and perspectives,” Mech. Syst. Signal Proc., vol. 72–73, pp. 2–31, May 2016. doi: 10.1016/j.ymssp.2015.11.008
    [30]
    A. Soualhi, H. Razik, G. Clerc, and D. D. Doan, “Prognosis of bearing failures using hidden Markov models and the adaptive neuro-fuzzy inference system,” IEEE Trans. Ind. Electron., vol. 61, no. 6, pp. 2864–2874, Jun. 2014. doi: 10.1109/TIE.2013.2274415
    [31]
    R. Khelif, B. Chebel-Morello, S. Malinowski, E. Laajili, F. Fnaiech, and N. Zerhouni, “Direct remaining useful life estimation based on support vector regression,” IEEE Trans. Ind. Electron., vol. 64, no. 3, pp. 2276–2285, Mar. 2017. doi: 10.1109/TIE.2016.2623260
    [32]
    Z. G. Tian, “An artificial neural network method for remaining useful life prediction of equipment subject to condition monitoring,” J. Intell. Manuf., vol. 23, no. 2, pp. 227–237, Apr. 2012. doi: 10.1007/s10845-009-0356-9
    [33]
    L. Saidi, J. Ben Ali, E. Bechhoefer, and M. Benbouzid, “Wind turbine high-speed shaft bearings health prognosis through a spectral kurtosis-derived indices and SVR,” Appl. Acoust., vol. 120, pp. 1–8, Mar. 2017. doi: 10.1016/j.apacoust.2017.01.005
    [34]
    J. Carreira and A. Zisserman, “Quo Vadis, action recognition? A new model and the kinetics dataset,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Honolulu, USA, 2017, pp. 4724–4733.
    [35]
    X. Z. Zhu, Y. J. Wang, J. F. Dai, L. Yuan, and Y. C. Wei, “Flow-guided feature aggregation for video object detection,” in Proc. IEEE Int. Conf. Computer Vision, Venice, Italy, 2017, pp. 408–417.
    [36]
    R. B. Jin, G. S. Lin, C. Y. Wen, J. L. Wang, and F. Y. Liu, “Feature flow: In-network feature flow estimation for video object detection,” Pattern Recogn., vol. 122, p. 108323, Feb. 2022.
    [37]
    M. D. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks,” in Proc. 13th European Conf. Computer Vision, Zurich, Switzerland, 2014, pp. 818–833.
    [38]
    S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in Proc. 32nd Int. Conf. Machine Learning, Lille, France, 2015, pp. 448–456.
    [39]
    A. Saxena, K. Goebel, D. Simon, and N. Eklund, “Damage propagation modeling for aircraft engine run-to-failure simulation,” in Proc. Int. Conf. Prognostics and Health Management, Denver, USA, 2008, pp. 1–9.
    [40]
    S. Behera and R. Misra, “Generative adversarial networks based remaining useful life estimation for IIoT,” Comput. Electr. Eng., vol. 92, p. 107195, Jun. 2021.
    [41]
    C. Zhang, P. Lim, A. K. Qin, and K. C. Tan, “Multiobjective deep belief networks ensemble for remaining useful life estimation in prognostics,” IEEE Trans. Neural Netw. Learn. Syst., vol. 28, no. 10, pp. 2306–2318, Oct. 2017. doi: 10.1109/TNNLS.2016.2582798
    [42]
    H. Y. Lu, L. Jin, X. Luo, B. L. Liao, D. S. Guo, and L. Xiao, “RNN for solving perturbed time-varying underdetermined linear system with double bound limits on residual errors and state variables,” IEEE Trans. Ind. Inf., vol. 15, no. 11, pp. 5931–5942, Nov. 2019. doi: 10.1109/TII.2019.2909142
    [43]
    L. Xin, Y. Yuan, M. C. Zhou, Z. G. Liu, and M. S. Shang, “Non-negative latent factor model based on β-divergence for recommender systems,” IEEE Trans. Syst. Man Cybern. Syst., vol. 51, no. 8, pp. 4612–4623, Aug. 2021. doi: 10.1109/TSMC.2019.2931468
    [44]
    X. Luo, Z. D. Wang, and M. S. Shang, “An instance-frequency-weighted regularization scheme for non-negative latent factor analysis on high-dimensional and sparse data,” IEEE Trans. Syst. Man Cybern. Syst., vol. 51, no. 6, pp. 3522–3532, Jun. 2021. doi: 10.1109/TSMC.2019.2930525

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(14)  / Tables(7)

    Article Metrics

    Article views (1767) PDF downloads(233) Cited by()

    Highlights

    • A new convolutional neural network is proposed for machine remaining useful life prediction
    • A series of important principles are developed for enhancing the performance of CNNs on the RUL prediction
    • A novel position encoding scheme is proposed for the RUL prediction
    • State-of-the-art performances have been achieved on the C-MAPSS benchmark dataset

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return