IEEE/CAA Journal of Automatica Sinica
Citation: | Y. Li, X. Wang, Z. He, Z. Wang, K. Cheng, S. Ding, Y. Fan, X. Li, Y. Niu, S. Xiao, Z. Hao, B. Gao, and H. Wu, “Industry-oriented detection method of PCBA defects using semantic segmentation models,” IEEE/CAA J. Autom. Sinica, vol. 11, no. 6, pp. 1438–1446, Jun. 2024. doi: 10.1109/JAS.2024.124422 |
Automated optical inspection (AOI) is a significant process in printed circuit board assembly (PCBA) production lines which aims to detect tiny defects in PCBAs. Existing AOI equipment has several deficiencies including low throughput, large computation cost, high latency, and poor flexibility, which limits the efficiency of online PCBA inspection. In this paper, a novel PCBA defect detection method based on a lightweight deep convolution neural network is proposed. In this method, the semantic segmentation model is combined with a rule-based defect recognition algorithm to build up a defect detection framework. To improve the performance of the model, extensive real PCBA images are collected from production lines as datasets. Some optimization methods have been applied in the model according to production demand and enable integration in lightweight computing devices. Experiment results show that the production line using our method realizes a throughput more than three times higher than traditional methods. Our method can be integrated into a lightweight inference system and promote the flexibility of AOI. The proposed method builds up a general paradigm and excellent example for model design and optimization oriented towards industrial requirements.
[1] |
G. Acciani, G. Brunetti, and G. Fornarelli, “Application of neural networks in optical inspection and classification of solder joints in surface mount technology,” IEEE Trans. Ind. Inf., vol. 2, no. 3, pp. 200–209, Aug. 2006. doi: 10.1109/TII.2006.877265
|
[2] |
K. Sundaraj, “PCB inspection for missing or misaligned components using background subtraction,” WSEAS Trans. Inform. Sci. Appl., vol. 6, no. 5, pp. 778–787, May 2009.
|
[3] |
H. Hagi, Y. Iwahori, S. Fukui, Y. Adachi, and M. K. Bhuyan, “Defect classification of electronic circuit board using SVM based on random sampling,” Procedia Comput. Sci., vol. 35, pp. 1210–1218, Sep. 2014. doi: 10.1016/j.procs.2014.08.218
|
[4] |
R. Ding, L. Dai, G. Li, and H. Liu, “TDD-Net: A tiny defect detection network for printed circuit boards,” CAAI Trans. Intell. Technol., vol. 4, no. 2, pp. 110–116, Jun. 2019. doi: 10.1049/trit.2019.0019
|
[5] |
B. Hu and J. Wang, “Detection of PCB surface defects with improved faster-RCNN and feature pyramid network,” IEEE Access, vol. 8, pp. 108335–108345, Jun. 2020. doi: 10.1109/ACCESS.2020.3001349
|
[6] |
L. H. De S. Silva, G. O. De A. Azevedo, B. J. T. Fernandes, B. L. D. Bezerra, E. B. Lima, and S. C. Oliveira, “Automatic optical inspection for defective PCB detection using transfer learning,” in Proc. IEEE Latin American Conf. Computational Intelligence, Guayaquil, Ecuador, 2019, pp. 1–6.
|
[7] |
Y. Liu, B. Jiang, and J. Xu, “Axial assembled correspondence network for few-shot semantic segmentation,” IEEE/CAA J. Autom. Sinica, vol. 10, no. 3, pp. 711–721, Mar. 2023. doi: 10.1109/JAS.2022.105863
|
[8] |
Y. Lin, Z. Xu, D. Chen, Z. Ai, Y. Qiu, and Y. Yuan, “Wood crack detection based on data-driven semantic segmentation network,” IEEE/CAA J. Autom. Sinica, vol. 10, no. 6, pp. 1510–1512, Jun. 2023. doi: 10.1109/JAS.2023.123357
|
[9] |
M. H. Annaby, Y. M. Fouda, and M. A. Rushdi, “Improved normalized cross-correlation for defect detection in printed-circuit boards,” IEEE Trans. Semicond. Manuf., vol. 32, no. 2, pp. 199–211, May 2019. doi: 10.1109/TSM.2019.2911062
|
[10] |
F. Li, F. Li, and Q. G. Xi, “DefectNet: Toward fast and effective defect detection,” IEEE Trans. Instrum. Meas., vol. 70, p. 2507109, Mar. 2021.
|
[11] |
J. Luo, Z. Yang, S. Li, and Y. Wu, “FPCB surface defect detection: A decoupled two-stage object detection framework,” IEEE Trans. Instrum. Meas., vol. 70, p. 5012311, Jun. 2021.
|
[12] |
J. M. Runji and C. Y. Lin, “Markerless cooperative augmented reality-based smart manufacturing double-check system: Case of safe PCBA inspection following automatic optical inspection,” Robot. Comput. Integr. Manuf., vol. 64, p. 101957, Aug. 2020. doi: 10.1016/j.rcim.2020.101957
|
[13] |
S. Gao, T. Qiu, A. Huang, G. Wang, and J. Yu, “Electronic components detection for PCBA based on a tailored YOLOv3 network with image pre-processing,” in Proc. 17th Int. Conf. Automation Science and Engineering, Lyon, France, 2021, pp. 1435–1440.
|
[14] |
D. K. Bonello, Y. Iano, and U. B. Neto, “A novel approach to the PCBAs defects detection using background algorithm,” Int. J. Res. Electron. Comput. Eng., vol. 8, no. 1, pp. 308–315, Jan.–Mar. 2020.
|
[15] |
Z. Lan, Y. Hong, and Y. Li, “An improved YOLOv3 method for PCB surface defect detection,” in Proc. IEEE Int. Conf. Power Electronics, Computer Applications, Shenyang, China, 2021, pp. 1009–1015.
|
[16] |
Y. T. Li, P. Kuo, and J. I. Guo, “Automatic industry PCB board DIP process defect detection with deep ensemble method,” in Proc. 29th Int. Symp. Industrial Electronics, Delft, Netherlands, 2020, pp. 453–459.
|
[17] |
Y. Li, P. Kuo, and J. Guo, “Automatic industry PCB board DIP process defect detection system based on deep ensemble self-adaption method,” IEEE Trans. Compon. Packag. Manuf. Technol., vol. 11, no. 2, pp. 312–323, Feb. 2021. doi: 10.1109/TCPMT.2020.3047089
|
[18] |
R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Columbus, USA, 2014, pp. 580–587.
|
[19] |
K. He, X. Zhang, S. Ren, and J. Sun, “Spatial pyramid pooling in deep convolutional networks for visual recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 9, pp. 1904–1916, Sep. 2015. doi: 10.1109/TPAMI.2015.2389824
|
[20] |
S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 6, pp. 1137–1149, Jun. 2017. doi: 10.1109/TPAMI.2016.2577031
|
[21] |
K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask R-CNN,” in Proc. IEEE Int. Conf. Computer Vision, Venice, Italy, 2017, pp. 2980–2988.
|
[22] |
E. Shelhamer, J. Long, and T. Darrell, “Fully convolutional networks for semantic segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 4, pp. 640–651, Apr. 2017. doi: 10.1109/TPAMI.2016.2572683
|
[23] |
O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” in Proc. 18th Int. Conf. Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 2015, pp. 234–241.
|
[24] |
K. Han, R. S. Rezende, B. Ham, K. Y. K. Wong, M. Cho, C. Schmid, and J. Ponce, “SCNet: Learning semantic correspondence,” in Proc. Int. Conf. Computer Vision, Venice, Italy, 2017, pp. 1849-1858.
|
[25] |
V. Badrinarayanan, A. Handa, and R. Cipolla, “SegNet: A deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling,” arXiv preprint arXiv: 1505.07293, 2015.
|
[26] |
L. C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 4, pp. 834–848, Apr. 2018. doi: 10.1109/TPAMI.2017.2699184
|