Keywords
Welding path programming
Machine vision
Multi-view reconstruction
U-Net segmentation
How to Cite
Abstract
Manual programming of robotic welding paths remains a limiting factor in flexible and small-batch production, where workpiece geometries and repair trajectories may vary between tasks. Teach-pendant programming in collaborative robotic welding stations is time-consuming and operator-dependent, while CAD-based offline programming, although common in conventional industrial robotics, is not always available or suitable for variable geometries, or uncertain repair operations. This paper presents a vision-assisted method for digitalising operator-defined welding paths and reconstructing them in the robot task space. The operator manually indicates the desired trajectory on the workpiece surface, which is then acquired from multiple camera viewpoints. A U-Net neural network is used to segment the path, while AprilTag fiducial markers support spatial referencing, projective reconstruction and multi-view triangulation. Experimental validation showed an average three-dimensional path reconstruction error of 0.98 mm. The proposed workflow provides a practical step towards reducing manual programming effort in robotic welding applications requiring operator-defined paths.
References
Brito, T., Queiroz, J., Piardi, L., Fernandes, L., Lima, J., & Leitao, P. (2020). A machine learning approach for collaborative robot smart manufacturing inspection for quality control systems. Procedia Manufacturing, 51, 11–18. https://doi.org/10.1016/j.promfg.2020.10.003
Chmielowiec, A. (2025). Utilization of neural networks and u-net architecture for cladding area detection for collaborative robots. In J. Machado, J. Trojanowska, F. Soares, P. Rea, S. Butdee, & B. Gramescu (Eds.), Innovations in mechatronics engineering iv (pp. 253–264). Cham: Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-94223-5_23
Chmielowiec, A., Koc, M., Łysiak, K., & Szuba, M. (2025). Automatic cladding trajectory generation using polygon triangulation and maximal alternating paths. Technologia i Automatyzacja Montażu, 130 (4/2025), 61–70. https://doi.org/10.7862/tiam.2025.4.6
Chmielowiec, A., Łysiak, K., & Vilis, J. (2024). Development of a Basic CAM Processor for a Collaborative Robot for Workshop Automation. Technologia i Automatyzacja Montażu, 123 (1), 37-45. https://doi.org/10.7862/tiam.2024.1.6
Forni, T., Mattera, G., Vozza, M., & Nele, L. (2025). Vision-based defect localisation and automated planning for robotic spray coating systems. Journal of the Brazilian Society of Mechanical Sciences and Engineering, 47 (9), 424. https://doi.org/10.1007/s40430-025-05746-z
Gómez-Espinosa, A., Rodrıguez-Suarez, J., Cuan-Urquizo, E., & Cabello, R., J.A.E.and Swenson. (2021). Colored 3d path extraction based on depth-rgb sensor for welding robot trajectory generation. Automation, 2 (4), 252–265. https://doi.org/10.3390/automation2040016
He, Z., & Liu, Q. (2020). Deep regression neural network for industrial surface defect detection. IEEE Access, 8, 35583–35591. https://doi.org/10.1109/ACCESS.2020.2975030
Iglhaut, J., Cabo, C., Puliti, S., Piermattei, L., O’Connor, J., & Rosette, J. (2019). Structurefrom motion photogrammetry in forestry: A review. Current Forestry Reports, 5 (3), 155–168. https://doi.org/10.1007/s40725-019-00094-3
Kalaitzakis, M., Cain, B., Carroll, S., Ambrosi, A., Whitehead, C., & Vitzilaios, N. (2021). Fiducial markers for pose estimation: Overview, applications and experimental comparison of the artag, apriltag, aruco and stag markers. Journal of Intelligent & Robotic Systems, 101 (4), 71. https://doi.org/10.1007/s10846-020-01307-9
Kohut, P., & Skop, K. (2024). Vision systems for a ur5 cobot on a quality control robotic station. Applied Sciences, 14 (20), 9469. https://doi.org/10.3390/app14209469
Li, A., Li, X., & Ma, X. (2024). Residual dual u-shape networks with improved skip connections for cloud detection. IEEE Geoscience and Remote Sensing Letters, 21, 1–5. https://doi.org/10.1109/LGRS.2023.3337860
Li, X., Ma, J., Yao, S., Chi, G., & Zhang, G. (2025). A dual-modal robot welding trajectory generation scheme for motion based on stereo vision and deep learning. Materials, 18 (11). https://doi.org/10.3390/ma18112593
Manoli, E., Mattera, G., Caggiano, A., Marzano, A., & Nele, L. (2026). From digital twins to immersive manufacturing: Xr and gesture-based control for enhancing human–robot collaboration. IET Collaborative Intelligent Manufacturing, 8 (1), e70055. https://doi.org/10.1049/cim2.70055
Manta-Costa, A., Araujo, S., Peres, R., & Barata, J. (2024). Machine learning applications in manufacturing - challenges, trends, and future directions. IEEE Open Journal of the Industrial Electronics Society, 5, 1085-1103. https://doi.org/10.1109/OJIES.2024.3431240
Matheson, E., Minto, R., Zampieri, E., Faccio, M., & Rosati, G. (2019). Human–robot collaboration in manufacturing applications: a review. Robotics, 8 (4), 100. https://doi.org/10.3390/robotics8040100
Ona, E. d., Rodrıguez-Martin, M., Rodrıguez-Gonzalvez, P., Mora, R., & Gonzalez-Aguilera, D. (2022). Weldmap: A photogrammetric suite applied to the inspection of welds. Applied Sciences, 12 (5). https://doi.org/10.3390/app12052553
Park, H., Shiratori, T., Matthews, I., & Sheikh, Y. (2015). 3d trajectory reconstruction under perspective projection. International Journal of Computer Vision, 115 (2), 115–135. https://doi.org/10.1007/s11263-015-0804-2
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In N. Navab, J. Hornegger, W. Wells, & A. Frangi (Eds.), Medical image computing and computer-assisted intervention – miccai 2015: 18th international conference, munich, germany, october 5-9, 2015, proceedings, part iii 18 (pp. 234–241). Springer. https://doi.org/10.1007/978-3-319-24574-4_28
Singh, S., & Desai, K. (2023). Automated surface defect detection framework using machine vision and convolutional neural networks. Journal of Intelligent Manufacturing, 34 (4), 1995–2011. https://doi.org/10.1007/s10845-021-01878-w
Tao, X., Zhang, D., Ma, W., Liu, X., & Xu, D. (2018). Automatic metallic surface defect detection and recognition with convolutional neural networks. Applied Sciences, 8 (9), 1575. https://doi.org/10.3390/app8091575
Vladov, S., Bulakh, M., Baranovskyi, D., Sokurenko, V., Muzychuk, O., & Vysotska, V. (2025). Helicopter turboshaft engines combustion chamber monitoring neural network method. Measurement, 242, 116267. https://doi.org/10.1016/j.measurement.2024.116267
Vladov, S., Yakovliev, R., Bulakh, M., & Vysotska, V. (2024). Neural network approximation of helicopter turboshaft engine parameters for improved efficiency. Energies, 17 (9), 2233. https://doi.org/10.3390/en17092233
Wang, X., Xie, Z., Zhou, X., Gao, J., Li, F., & Gu, X. (2022). Adaptive path planning for the gantry welding robot system. Journal of Manufacturing Processes, 81, 386–395. https://doi.org/10.1016/j.jmapro.2022.07.005
Wang, X., Zhou, X., Xia, Z., & Gu, X. (2021). A survey of welding robot intelligent path optimization. Journal of Manufacturing Processes, 63, 14–23. https://doi.org/10.1016/j.jmapro.2020.04.085
Wei, Y., Kang, L., Yang, B., & Wu, L. (2013). Applications of structure from motion: a survey. Journal of Zhejiang University-SCIENCE C (Computers & Electronics), 14 (7), 486–494. https://doi.org/10.1631/jzus.CIDE1302
Williams, C., Falck, F., Deligiannidis, G., Holmes, C., Doucet, A., & Syed, S. (2023). A unified framework for u-net design and analysis. Advances in Neural Information Processing Systems, 36, 27745–27782
Xing, J., & Jia, M. (2021). A convolutional neural network-based method for workpiece surface defect detection. Measurement, 176, 109185. https://doi.org/10.1016/j.measurement.2021.109185
Yang, X., Zhou, Z., Sørensen, J., Christensen, C., Unalan, M., & Zhang, X. (2023). Automation of sme production with a cobot system powered by learning-based vision. Robotics and Computer-Integrated Manufacturing, 83, 102564. https://doi.org/10.1016/j.rcim.2023.102564
Zheng, Y. (2021). Intelligent and automatic inspection, reconstruction and process planning methods for remanufacturing and repair (Doctoral dissertation, University of Alberta). https://doi.org/10.7939/r3-5qqk-8408