A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Advanced Search
Volume 10 Issue 6
Jun.  2023

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 15.3, Top 1 (SCI Q1)
    CiteScore: 23.5, Top 2% (Q1)
    Google Scholar h5-index: 77， TOP 5
Turn off MathJax
Article Contents
Article Navigation >  IEEE/CAA Journal of Automatica Sinica  > 2023  >  10(6): 1445-1461
X. Y. Jiang, X. Y. Kong, and Z. Q. Ge, “Augmented industrial data-driven modeling under the curse of dimensionality,” IEEE/CAA J. Autom. Sinica, vol. 10, no. 6, pp. 1445–1461, Jun. 2023. doi: 10.1109/JAS.2023.123396
Citation: X. Y. Jiang, X. Y. Kong, and Z. Q. Ge, “Augmented industrial data-driven modeling under the curse of dimensionality,” IEEE/CAA J. Autom. Sinica, vol. 10, no. 6, pp. 1445–1461, Jun. 2023. doi: 10.1109/JAS.2023.123396
shu

Augmented Industrial Data-Driven Modeling Under the Curse of Dimensionality

doi: 10.1109/JAS.2023.123396
Funds:  This work was supported in part by the National Natural Science Foundation of China (NSFC) (92167106, 61833014) and Key Research and Development Program of Zhejiang Province (2022C01206)
More Information
    Abstract
  • The curse of dimensionality refers to the problem of increased sparsity and computational complexity when dealing with high-dimensional data. In recent years, the types and variables of industrial data have increased significantly, making data-driven models more challenging to develop. To address this problem, data augmentation technology has been introduced as an effective tool to solve the sparsity problem of high-dimensional industrial data. This paper systematically explores and discusses the necessity, feasibility, and effectiveness of augmented industrial data-driven modeling in the context of the curse of dimensionality and virtual big data. Then, the process of data augmentation modeling is analyzed, and the concept of data boosting augmentation is proposed. The data boosting augmentation involves designing the reliability weight and actual-virtual weight functions, and developing a double weighted partial least squares model to optimize the three stages of data generation, data fusion, and modeling. This approach significantly improves the interpretability, effectiveness, and practicality of data augmentation in the industrial modeling. Finally, the proposed method is verified using practical examples of fault diagnosis systems and virtual measurement systems in the industry. The results demonstrate the effectiveness of the proposed approach in improving the accuracy and robustness of data-driven models, making them more suitable for real-world industrial applications.

     

    • FullText(HTML)
  • loading
    • References(49)
  • [1]
    J. C. Qian, L. Jiang, and Z. H. Song, “Locally linear back-propagation based contribution for nonlinear process fault diagnosis,” IEEE/CAA J. Autom. Sinica, vol. 7, no. 3, pp. 764–775, May 2020. doi: 10.1109/JAS.2020.1003147
    [2]
    A. White, A. Karimoddini, and M. Karimadini, “Resilient fault diagnosis under imperfect observations–a need for industry 4.0 era,” IEEE/CAA J. Autom. Sinica, vol. 7, no. 5, pp. 1279–1288, Sept. 2020. doi: 10.1109/JAS.2020.1003333
    [3]
    R. B. Jin, M. Wu, K. Y. Wu, K. Z. Gao, Z. H. Chen, and X. L. Li, “Position encoding based convolutional neural networks for machine remaining useful life prediction,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 8, pp. 1427–1439, Aug. 2022. doi: 10.1109/JAS.2022.105746
    [4]
    Z. Y. Yang and Z. Q. Ge, “On paradigm of industrial big data analytics: From evolution to revolution,” IEEE Trans. Ind. Inf., vol. 18, no. 12, pp. 8373–8388, Dec. 2022. doi: 10.1109/TII.2022.3190394
    [5]
    X. Y. Kong, X. Y. Jiang, B. X. Zhang, J. S. Yuan, and Z. Q. Ge, “Latent variable models in the era of industrial big data: Extension and beyond,” Annu. Rev. Control, vol. 54, pp. 167–199, 2022.
    [6]
    X. Luo, M. C. Zhou, S. Li, L. Hu, and M. S. Shang, “Non-negativity constrained missing data estimation for high-dimensional and sparse matrices from industrial applications,” IEEE Trans. Cybern., vol. 50, no. 5, pp. 1844–1855, May 2020. doi: 10.1109/TCYB.2019.2894283
    [7]
    K. K. Huang, Y. M. Wu, C. Wang, Y. F. Xie, C. H. Yang, and W. H. Gui, “A projective and discriminative dictionary learning for high-dimensional process monitoring with industrial applications,” IEEE Trans. Ind. Inf., vol. 17, no. 1, pp. 558–568, Jan. 2021. doi: 10.1109/TII.2020.2992728
    [8]
    B. B. Shen, L. Yao, and Z. Q. Ge, “Predictive modeling with multiresolution pyramid VAE and industrial soft sensor applications,” IEEE Trans. Cybern., 2022. DOI: 10.1109/TCYB.2022.3143613
    [9]
    L. Yao, B. B. Shen, L. L. Cui, J. H. Zheng, and Z. Q. Ge, “Semi-supervised deep dynamic probabilistic latent variable model for multimode process soft sensor application,” IEEE Trans. Ind. Inf., vol. 19, no. 4, pp. 6056–6068, Apr. 2023. doi: 10.1109/TII.2022.3183211
    [10]
    A. Glowacz, “Acoustic based fault diagnosis of three-phase induction motor,” Appl. Acoust., vol. 137, pp. 82–89, Aug. 2018. doi: 10.1016/j.apacoust.2018.03.010
    [11]
    K. Muhammad, T. Hussain, J. Del Ser, V. Palade, and V. H. C. De Albuquerque, “DeePreS: A deep learning-based video summarization strategy for resource-constrained industrial surveillance scenarios,” IEEE Trans. Ind. Inf., vol. 16, no. 9, pp. 5938–5947, Sept. 2020. doi: 10.1109/TII.2019.2960536
    [12]
    X. Y. Jiang and Z. Q. Ge, “Augmented multidimensional convolutional neural network for industrial soft sensing,” IEEE Trans. Instrum. Meas., vol. 70, p. 2508410, Apr. 2021.
    [13]
    D. L. Zheng, L. Zhou, and Z. H. Song, “Kernel generalization of multi-rate probabilistic principal component analysis for fault detection in nonlinear process,” IEEE/CAA J. Autom. Sinica, vol. 8, no. 8, pp. 1465–1476, Aug. 2021. doi: 10.1109/JAS.2021.1004090
    [14]
    N. Altman and M. Krzywinski, “The curse(s) of dimensionality,” Nat. Methods, vol. 15, no. 6, pp. 399–400, May 2018. doi: 10.1038/s41592-018-0019-x
    [15]
    H. Y. Liu, M. C. Zhou, and Q. Liu, “An embedded feature selection method for imbalanced data classification,” IEEE/CAA J. Autom. Sinica, vol. 6, no. 3, pp. 703–715, May 2019. doi: 10.1109/JAS.2019.1911447
    [16]
    J. H. Wang, L. Y. Qiao, Y. Q. Ye, and Y. Q. Chen, “Fractional envelope analysis for rolling element bearing weak fault feature extraction,” IEEE/CAA J. Autom. Sinica, vol. 4, no. 2, pp. 353–360, Apr. 2017. doi: 10.1109/JAS.2016.7510166
    [17]
    C. Shorten and T. M. Khoshgoftaar, “A survey on image data augmentation for deep learning,” J. Big Data, vol. 6, no. 1, p. 60, Jul. 2019. doi: 10.1186/s40537-019-0197-0
    [18]
    Y. Grandvalet, S. Canu, and S. Boucheron, “Noise injection: Theoretical prospects,” Neural Comput., vol. 9, no. 5, pp. 1093–1108, Jul. 1997. doi: 10.1162/neco.1997.9.5.1093
    [19]
    N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. Kegelmeyer, “SMOTE: Synthetic minority over-sampling technique,” J. Artif. Intell. Res., vol. 16, no. 1, pp. 321–357, Jan. 2002.
    [20]
    H. Y. Zhang, M. Cissé, Y. N. Dauphin, and D. Lopez-Paz, “mixup: Beyond empirical risk minimization,” in Proc. 6th Int. Conf. Learning Representations, Vancouver, Canada, 2017.
    [21]
    H. Inoue, “Data augmentation by pairing samples for images classification,” arXiv preprint arXiv: 1801.02929, 2018.
    [22]
    F. N. Hatamian, N. Ravikumar, S. Vesal, F. P. Kemeth, M. Struck, and A. Maier, “The effect of data augmentation on classification of atrial fibrillation in short single-lead ECG signals using deep neural networks,” in IEEE Int. Conf. Acoustics, Speech and Signal Processing, Barcelona, Spain, 2020, pp. 1264–1268.
    [23]
    I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Proc. 27th Int. Conf. Neural Information Processing Systems, Montreal, Canada, 2014, pp. 2672–2680.
    [24]
    X. Y. Jiang and Z. Q. Ge, “Data augmentation classifier for imbalanced fault classification,” IEEE Trans. Automation Science and Engineering, vol. 18, no. 3, pp. 1206–1217, Jul. 2021. doi: 10.1109/TASE.2020.2998467
    [25]
    L. Li, S. K. Damarla, Y. L. Wang, and B. Huang, “A Gaussian mixture model based virtual sample generation approach for small datasets in industrial processes,” Inf. Sci., vol. 581, pp. 262–277, Dec. 2021. doi: 10.1016/j.ins.2021.09.014
    [26]
    X. Y. Wang, Z. Y. Chu, B. K. Han, J. R. Wang, G. W. Zhang, and X. X. Jiang, “A novel data augmentation method for intelligent fault diagnosis under speed fluctuation condition,” IEEE Access, vol. 8, pp. 143383–143396, Aug. 2020. doi: 10.1109/ACCESS.2020.3014340
    [27]
    A. Fujishiro, Y. Nagamura, T. Usami, and M. Inoue, “Minimizing convolutional neural network training data with proper data augmentation for inline defect classification,” IEEE Trans. Semicond. Manuf., vol. 34, no. 3, pp. 333–339, Aug. 2021. doi: 10.1109/TSM.2021.3074456
    [28]
    X. Y. Jiang and Z. Q. Ge, “RAGAN: Regression attention generative adversarial networks,” IEEE Trans. Artif. Intell., 2022. DOI: 10.1109/TAI.2022.3209956
    [29]
    V. Vapnik, “Principles of risk minimization for learning theory,” in Proc. 4th Int. Conf. Neural Information Processing Systems, Denver, USA, 1991, pp. 831–838.
    [30]
    D. H. Wolpert and W. G. Macready, “No free lunch theorems for optimization,” IEEE Trans. Evol. Comput., vol. 1, no. 1, pp. 67–82, Apr. 1997. doi: 10.1109/4235.585893
    [31]
    M. Hutter, “On the existence and convergence of computable universal priors,” in Proc. 14th Int. Conf. Algorithmic Learning Theory, Sapporo, Japan, 2003, pp. 298–312.
    [32]
    P. Bühlmann and S. Van De Geer, Statistics for High-Dimensional Data: Methods, Theory and Applications. Berlin, Germany: Springer, 2011.
    [33]
    N. S. Altman, “An introduction to kernel and nearest-neighbor nonparametric regression,” Am. Stat., vol. 46, no. 3, pp. 175–185, Feb. 1992.
    [34]
    I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, USA: MIT Press, 2016.
    [35]
    X. F. He, D. Cai, S. C. Yan, and H.-J. Zhang, “Neighborhood preserving embedding,” in Proc. 10th IEEE Int. Conf. Computer Vision, Beijing, China, 2005, pp. 1208–1213.
    [36]
    Z. Q. Ge and Z. H. Song, “A comparative study of just-in-time-learning based methods for online soft sensor modeling,” Chemom. Intell. Lab. Syst., vol. 104, no. 2, pp. 306–317, Dec. 2010. doi: 10.1016/j.chemolab.2010.09.008
    [37]
    M. Balasubramanian and E. L. Schwartz, “The isomap algorithm and topological stability,” Science, vol. 295, no. 5552, p. 7, Jan. 2002. doi: 10.1126/science.295.5552.7a
    [38]
    A. Halevy, Norvig, and F. Pereira, “The unreasonable effectiveness of data,” IEEE Intell. Syst., vol. 24, no. 2, pp. 8–12, Mar.–Apr. 2009. doi: 10.1109/MIS.2009.36
    [39]
    M. J. Kearns and U. Vazirani, An Introduction to Computational Learning Theory. Cambridge, USA: MIT Press, 1994.
    [40]
    A. Blumer, A. Ehrenfeucht, D. Haussler, and M. K. Warmuth, “Learnability and the vapnik-chervonenkis dimension,” J. ACM, vol. 36, no. 4, pp. 929–965, Oct. 1989. doi: 10.1145/76359.76371
    [41]
    C. Sun, A. Shrivastava, S. Singh, and A. Gupta, “Revisiting unreasonable effectiveness of data in deep learning era,” in IEEE Int. Conf. Computer Vision, Venice, Italy, 2017, pp. 843–852.
    [42]
    J. Cho, K. Lee, E. Shin, G. Choy, and S. Do, “How much data is needed to train a medical image deep learning system to achieve necessary high accuracy?” arXiv preprint arXiv: 1511.06348, 2016.
    [43]
    J. Hestness, S. Narang, N. Ardalani, G. Diamos, H. Jun, H. Kianinejad, M. M. A. Patwary, Y. Yang, and Y. Q. Zhou, “Deep learning scaling is predictable, empirically,” arXiv preprint arXiv: 1712.00409, 2017.
    [44]
    X. Y. Jiang and Z. Q. Ge, “Improving the performance of just-in-time learning-based soft sensor through data augmentation,” IEEE Trans. Ind. Electron., vol. 69, no. 12, pp. 13716–13726, Dec. 2022. doi: 10.1109/TIE.2021.3139194
    [45]
    X. Y. Kong and Z. Q. Ge, “Deep PLS: A lightweight deep learning model for interpretable and efficient data analytics,” IEEE Trans. Neural Netw. Learn. Syst., 2022. DOI: 10.1109/TNNLS.2022.3154090
    [46]
    C. K. Williams and M. Seeger, “Using the Nyström method to speed up kernel machines,” in Proc. 13th Int. Conf. Neural Information Processing Systems, Denver, USA, 2000, pp. 661–667.
    [47]
    J. J. Downs and E. F. Vogel, “A plant-wide industrial process control problem,” Comput. Chem. Eng., vol. 17, no. 3, pp. 245–255, Mar. 1993. doi: 10.1016/0098-1354(93)80018-I
    [48]
    X. Y. Jiang and Z. Q. Ge, “Information fingerprint for secure industrial big data analytics,” IEEE Trans. Ind. Inf., vol. 18, no. 4, pp. 2641–2650, Apr. 2022. doi: 10.1109/TII.2021.3104056
    [49]
    X. Y. Jiang and Z. Q. Ge, “Attacks on data-driven process monitoring systems: Subspace transfer networks,” IEEE Trans. Artif. Intell., vol. 3, no. 3, pp. 470–484, Jun. 2022. doi: 10.1109/TAI.2022.3145335
    • Related Articles
    • Supplements(0)

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(23)  / Tables(4)

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (129) PDF downloads(8) Cited by()

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return