A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Volume 11 Issue 6
Jun.  2024

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 11.8, Top 4% (SCI Q1)
    CiteScore: 23.5, Top 2% (Q1)
    Google Scholar h5-index: 77, TOP 5
Turn off MathJax
Article Contents
Y. Li, X. Wang, Z. He, Z. Wang, K. Cheng, S.  Ding, Y. Fan, X. Li, Y. Niu, S. Xiao, Z. Hao, B. Gao, and  H. Wu,  “Industry-oriented detection method of PCBA defects using semantic segmentation models,” IEEE/CAA J. Autom. Sinica, vol. 11, no. 6, pp. 1438–1446, Jun. 2024. doi: 10.1109/JAS.2024.124422
Citation: Y. Li, X. Wang, Z. He, Z. Wang, K. Cheng, S.  Ding, Y. Fan, X. Li, Y. Niu, S. Xiao, Z. Hao, B. Gao, and  H. Wu,  “Industry-oriented detection method of PCBA defects using semantic segmentation models,” IEEE/CAA J. Autom. Sinica, vol. 11, no. 6, pp. 1438–1446, Jun. 2024. doi: 10.1109/JAS.2024.124422

Industry-Oriented Detection Method of PCBA Defects Using Semantic Segmentation Models

doi: 10.1109/JAS.2024.124422
Funds:  This work was supported in part by the IoT Intelligent Microsystem Center of Tsinghua University-China Mobile Joint Research Institute
More Information
  • Automated optical inspection (AOI) is a significant process in printed circuit board assembly (PCBA) production lines which aims to detect tiny defects in PCBAs. Existing AOI equipment has several deficiencies including low throughput, large computation cost, high latency, and poor flexibility, which limits the efficiency of online PCBA inspection. In this paper, a novel PCBA defect detection method based on a lightweight deep convolution neural network is proposed. In this method, the semantic segmentation model is combined with a rule-based defect recognition algorithm to build up a defect detection framework. To improve the performance of the model, extensive real PCBA images are collected from production lines as datasets. Some optimization methods have been applied in the model according to production demand and enable integration in lightweight computing devices. Experiment results show that the production line using our method realizes a throughput more than three times higher than traditional methods. Our method can be integrated into a lightweight inference system and promote the flexibility of AOI. The proposed method builds up a general paradigm and excellent example for model design and optimization oriented towards industrial requirements.

     

  • loading
  • [1]
    G. Acciani, G. Brunetti, and G. Fornarelli, “Application of neural networks in optical inspection and classification of solder joints in surface mount technology,” IEEE Trans. Ind. Inf., vol. 2, no. 3, pp. 200–209, Aug. 2006. doi: 10.1109/TII.2006.877265
    [2]
    K. Sundaraj, “PCB inspection for missing or misaligned components using background subtraction,” WSEAS Trans. Inform. Sci. Appl., vol. 6, no. 5, pp. 778–787, May 2009.
    [3]
    H. Hagi, Y. Iwahori, S. Fukui, Y. Adachi, and M. K. Bhuyan, “Defect classification of electronic circuit board using SVM based on random sampling,” Procedia Comput. Sci., vol. 35, pp. 1210–1218, Sep. 2014. doi: 10.1016/j.procs.2014.08.218
    [4]
    R. Ding, L. Dai, G. Li, and H. Liu, “TDD-Net: A tiny defect detection network for printed circuit boards,” CAAI Trans. Intell. Technol., vol. 4, no. 2, pp. 110–116, Jun. 2019. doi: 10.1049/trit.2019.0019
    [5]
    B. Hu and J. Wang, “Detection of PCB surface defects with improved faster-RCNN and feature pyramid network,” IEEE Access, vol. 8, pp. 108335–108345, Jun. 2020. doi: 10.1109/ACCESS.2020.3001349
    [6]
    L. H. De S. Silva, G. O. De A. Azevedo, B. J. T. Fernandes, B. L. D. Bezerra, E. B. Lima, and S. C. Oliveira, “Automatic optical inspection for defective PCB detection using transfer learning,” in Proc. IEEE Latin American Conf. Computational Intelligence, Guayaquil, Ecuador, 2019, pp. 1–6.
    [7]
    Y. Liu, B. Jiang, and J. Xu, “Axial assembled correspondence network for few-shot semantic segmentation,” IEEE/CAA J. Autom. Sinica, vol. 10, no. 3, pp. 711–721, Mar. 2023. doi: 10.1109/JAS.2022.105863
    [8]
    Y. Lin, Z. Xu, D. Chen, Z. Ai, Y. Qiu, and Y. Yuan, “Wood crack detection based on data-driven semantic segmentation network,” IEEE/CAA J. Autom. Sinica, vol. 10, no. 6, pp. 1510–1512, Jun. 2023. doi: 10.1109/JAS.2023.123357
    [9]
    M. H. Annaby, Y. M. Fouda, and M. A. Rushdi, “Improved normalized cross-correlation for defect detection in printed-circuit boards,” IEEE Trans. Semicond. Manuf., vol. 32, no. 2, pp. 199–211, May 2019. doi: 10.1109/TSM.2019.2911062
    [10]
    F. Li, F. Li, and Q. G. Xi, “DefectNet: Toward fast and effective defect detection,” IEEE Trans. Instrum. Meas., vol. 70, p. 2507109, Mar. 2021.
    [11]
    J. Luo, Z. Yang, S. Li, and Y. Wu, “FPCB surface defect detection: A decoupled two-stage object detection framework,” IEEE Trans. Instrum. Meas., vol. 70, p. 5012311, Jun. 2021.
    [12]
    J. M. Runji and C. Y. Lin, “Markerless cooperative augmented reality-based smart manufacturing double-check system: Case of safe PCBA inspection following automatic optical inspection,” Robot. Comput. Integr. Manuf., vol. 64, p. 101957, Aug. 2020. doi: 10.1016/j.rcim.2020.101957
    [13]
    S. Gao, T. Qiu, A. Huang, G. Wang, and J. Yu, “Electronic components detection for PCBA based on a tailored YOLOv3 network with image pre-processing,” in Proc. 17th Int. Conf. Automation Science and Engineering, Lyon, France, 2021, pp. 1435–1440.
    [14]
    D. K. Bonello, Y. Iano, and U. B. Neto, “A novel approach to the PCBAs defects detection using background algorithm,” Int. J. Res. Electron. Comput. Eng., vol. 8, no. 1, pp. 308–315, Jan.–Mar. 2020.
    [15]
    Z. Lan, Y. Hong, and Y. Li, “An improved YOLOv3 method for PCB surface defect detection,” in Proc. IEEE Int. Conf. Power Electronics, Computer Applications, Shenyang, China, 2021, pp. 1009–1015.
    [16]
    Y. T. Li, P. Kuo, and J. I. Guo, “Automatic industry PCB board DIP process defect detection with deep ensemble method,” in Proc. 29th Int. Symp. Industrial Electronics, Delft, Netherlands, 2020, pp. 453–459.
    [17]
    Y. Li, P. Kuo, and J. Guo, “Automatic industry PCB board DIP process defect detection system based on deep ensemble self-adaption method,” IEEE Trans. Compon. Packag. Manuf. Technol., vol. 11, no. 2, pp. 312–323, Feb. 2021. doi: 10.1109/TCPMT.2020.3047089
    [18]
    R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Columbus, USA, 2014, pp. 580–587.
    [19]
    K. He, X. Zhang, S. Ren, and J. Sun, “Spatial pyramid pooling in deep convolutional networks for visual recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 9, pp. 1904–1916, Sep. 2015. doi: 10.1109/TPAMI.2015.2389824
    [20]
    S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 6, pp. 1137–1149, Jun. 2017. doi: 10.1109/TPAMI.2016.2577031
    [21]
    K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask R-CNN,” in Proc. IEEE Int. Conf. Computer Vision, Venice, Italy, 2017, pp. 2980–2988.
    [22]
    E. Shelhamer, J. Long, and T. Darrell, “Fully convolutional networks for semantic segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 4, pp. 640–651, Apr. 2017. doi: 10.1109/TPAMI.2016.2572683
    [23]
    O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” in Proc. 18th Int. Conf. Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 2015, pp. 234–241.
    [24]
    K. Han, R. S. Rezende, B. Ham, K. Y. K. Wong, M. Cho, C. Schmid, and J. Ponce, “SCNet: Learning semantic correspondence,” in Proc. Int. Conf. Computer Vision, Venice, Italy, 2017, pp. 1849-1858.
    [25]
    V. Badrinarayanan, A. Handa, and R. Cipolla, “SegNet: A deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling,” arXiv preprint arXiv: 1505.07293, 2015.
    [26]
    L. C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 4, pp. 834–848, Apr. 2018. doi: 10.1109/TPAMI.2017.2699184

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(10)  / Tables(2)

    Article Metrics

    Article views (55) PDF downloads(12) Cited by()

    Highlights

    • This paper proposes a novel defect detection method for complex PCBAs which is geared towards actual production lines. The method uses the CNN model to extract segmentation maps of raw PCBA images and detect tiny defects using visual geometric features, which can successfully detect defects in PCB with different designs
    • A semantic segmentation model combined with computer vision methods is proposed to build up a defect detection framework. Moreover, thanks to the rule-based defect recognition method, there is no need for defects data to be trained, which saves the cost of data collection and enhances the method's generalization capability
    • The proposed method can significantly improve the throughput of the PCBA production line with a lower false-call rate and missing detection rate, which shows a great improvement in production efficiency. By integrating with the edge intelligent system, the power consumption can be reduced by an order of magnitude
    • We design and implement a general paradigm for optimizing the PCBA detection model based on actual production objectives. With the increase of PCBA complexity, this paradigm can also provide us with a guide for adapting the detection model to new production lines
    • A PCBA data set for defect detection is built based on the images obtained in SMT production lines. The data set covers the main defect types in the actual production of PCBA and has been annotated manually

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return