Explainable AI: Objection Detection in Automated Vehicles using YOLO
DOI:
https://doi.org/10.59992//IJCI.2025.v4n10p1الكلمات المفتاحية:
Explainable AI، XAI، YOLO، KITTI، Grad-CAM، Eigen-CAM، Computer Vision، Autonomous Vehiclesالملخص
There is a recognized need for explainable artificial intelligence (XAI) in today's age as AIs are getting integrated more and more into our lives, from everyday tasks to huge fields where AI decisions are in a critical environment and could impact people's lives, XAI aims to make us understand the model behavior and decisions using developed methods, one such field is autonomous vehicles where there are many tasks to that uses different models, object detection in one of these tasks where the vehicles need to observe and assess the environment around them, several studies have shown the importance of XAI in the fields autonomous vehicles, and explained their actions and detections. However, true explainability has not yet been reached, with the complexity of Deep Neural Networks (DNN) and the tasks of AV, and the impact of performance and explainability motivates more studies of explainability. In this study, we train YOLOv11 and YOLOv12 models on the KITTI and a cyclist dataset and implement the explainability techniques Grad-CAM++ and Eigen-CAM to explain the decision behind them. The models showcased great performance, achieving 0.93 precision, 0.93 recall, 0.95 mAP@50, and 0.83 mAP@50-95 for YOLOv11 and similar results from YOLOv12. The heatmaps generated from Grad-cam and
Eigen-cam showcased where the model focuses on for the detections, overall explaining that this model will increase our trust and safety of AVs.
المراجع
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” 2012. [Online]. Available: http://code.google.com/p/cuda-convnet/.
[2] A. B. Arrieta et al., “Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI,” Information fusion, Oct. 2019, [Online]. Available: http://arxiv.org/abs/1910.10045.
[3] D. Gunning, M. Stefik, J. Choi, T. Miller, S. Stumpf, and G. Z. Yang, “XAI-Explainable artificial intelligence,” Sci Robot, vol. 4, no. 37, Dec. 2019, doi: 10.1126/scirobotics.aay7120.
[4] F. Rosique, P. J. Navarro, C. Fernández, and A. Padilla, “A systematic review of perception system and simulators for autonomous vehicles research,” Feb. 01, 2019, MDPI AG. doi: 10.3390/s19030648.
[5] V. Hassija et al., “Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence,” Jan. 01, 2024, Springer. doi: 10.1007/s12559-023-10179-8.
[6] P. P. Angelov, E. A. Soares, R. Jiang, N. I. Arnold, and P. M. Atkinson, “Explainable artificial intelligence: an analytical review,” Wiley Interdiscip Rev Data Min Knowl Discov, vol. 11, no. 5, Sep. 2021, doi: 10.1002/widm.1424.
[7] É. Zablocki, H. Ben-Younes, P. Pérez, and M. Cord, “Explainability of deep vision-based autonomous driving systems: Review and challenges,” ArXiv, Jan. 2021, [Online]. Available: http://arxiv.org/abs/2101.05307.
[8] S. Lundberg and S.-I. Lee, “A Unified Approach to Interpreting Model Predictions,” ArXiv, May 2017, [Online]. Available: http://arxiv.org/abs/1705.07874.
[9] M. T. Ribeiro, S. Singh, and C. Guestrin, “‘Why should i trust you?’ Explaining the predictions of any classifier,” in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Association for Computing Machinery, Aug. 2016, pp. 1135–1144. doi: 10.1145/2939672.2939778.
[10] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization,” IEEE Xplore, 2017, [Online]. Available: http://gradcam.cloudcv.org.
[11] V. Hassija et al., “Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence,” Jan. 01, 2024, Springer. doi: 10.1007/s12559-023-10179-8.
[12] M. Reda, A. Onsy, A. Ghanbari, and A. Y. Haikal, “Path planning algorithms in the autonomous driving system: A comprehensive review,” Apr. 01, 2024, Elsevier B.V. doi: 10.1016/j.robot.2024.104630.
[13] K. Rana, G. Gupta, P. Vaidya, and M. Khari, “The perception systems used in fully automated vehicles: a comparative analysis,” Multimed Tools Appl, 2023, doi: 10.1007/s11042-023-15090-w.
[14] Kamal Malik, Moolchand Sharma, Suman Deswal, Umesh Gupta, Deevyankar Agarwal, and Yahya Obaid Bakheet Al Shamsi, Explainable Artificial Intelligence for Autonomous Vehicles: Concepts, Challenges, and Applications. CRC press, 2024.
[15] “J3016_202104: Taxonomy and definitions for terms related to driving Automation Systems for On-Road Motor Vehicles - SAE International.,” https://www.sae.org/standards/content/j3016_202104/.
[16] A. Kuznietsov, B. Gyevnar, C. Wang, S. Peters, and S. V. Albrecht, “Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Review,” ArXiv, Feb. 2024, doi: 10.1109/TITS.2024.3474469.
[17] S. Atakishiyev, M. Salameh, H. Yao, and R. Goebel, “Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directions,” IEEE Access, vol. 12, pp. 101603–101625, 2024, doi: 10.1109/ACCESS.2024.3431437.
[18] A. Balasubramaniam and S. Pasricha, “Object Detection in Autonomous Vehicles: Status and Open Challenges,” ArXiv, 2022.
[19] M. Carranza-García, J. Torres-Mateo, P. Lara-Benítez, and J. García-Gutiérrez, “On the performance of one-stage and two-stage object detectors in autonomous vehicles using camera data,” Remote Sens (Basel), vol. 13, no. 1, pp. 1–23, Jan. 2021, doi: 10.3390/rs13010089.
[20] Z. Zou, K. Chen, Z. Shi, Y. Guo, and J. Ye, “Object Detection in 20 Years: A Survey,” ArXiv, May 2019, [Online]. Available: http://arxiv.org/abs/1905.05055
[21] J. Kim et al., “Toward explainable and advisable model for self-driving cars,” Applied AI Letters, vol. 2, no. 4, Dec. 2021, doi: 10.1002/ail2.56.
[22] J. Kim, S. Moon, A. Rohrbach, T. Darrell, and J. Canny, “Advisable Learning for Self-driving Vehicles by Internalizing Observation-to-Action Rules,” IEEE, 2020, [Online]. Available: https://github.
[23] Y. Xu et al., “Explainable Object-induced Action Decision for Autonomous Vehicles,” IEEE, 2020.
[24] C. Li, S. H. Chan, and Y.-T. Chen, “Who Make Drivers Stop? Towards Driver-centric Risk Assessment: Risk Object Identification via Causal Inference,” IEEE, Mar. 2020, [Online]. Available: http://arxiv.org/abs/2003.02425.
[25] H. Mankodiya, D. Jadav, R. Gupta, S. Tanwar, W. C. Hong, and R. Sharma, “OD-XAI: Explainable AI-Based Semantic Object Detection for Autonomous Vehicles,” Applied Sciences (Switzerland), vol. 12, no. 11, Jun. 2022, doi: 10.3390/app12115310.
[26] C.-H. Hsieh and Y.-H. Chang, “Improving DCP Haze Removal Scheme by Parameter Setting and Adaptive Gamma Correction,” 2021.
[27] V. S. Saravanarajan, R. C. Chen, C. H. Hsieh, and L. S. Chen, “Improving Semantic Segmentation Under Hazy Weather for Autonomous Vehicles Using Explainable Artificial Intelligence and Adaptive Dehazing Approach,” IEEE Access, vol. 11, pp. 38194–38207, 2023, doi: 10.1109/ACCESS.2023.3251728.
[28] M. Keser, G. Schwalbe, A. Nowzad, and A. Knoll, “Interpretable Model-Agnostic Plausibility Verification for 2D Object Detectors Using Domain-Invariant Concept Bottleneck Models,” IEEE, 2023.
[29] J. H. Sejr, P. Schneider-Kamp, and N. Ayoub, “Surrogate Object Detection Explainer (SODEx) with YOLOv4 and LIME,” Mach Learn Knowl Extr, vol. 3, no. 3, pp. 662–671, Sep. 2021, doi: 10.3390/make3030033.
[30] M. Alwateer, K. O. Aljuhani, A. Shaqrah, R. ElAgamy, G. Elmarhomy, and E. S. Atlam, “XAI-SALPAD: Explainable deep learning techniques for Saudi Arabia license plate automatic detection,” Alexandria Engineering Journal, vol. 109, pp. 578–590, Dec. 2024, doi: 10.1016/j.aej.2024.09.057.
[31] M. Moradi, K. Yan, D. Colwell, M. Samwald, and R. Asgari, “Model-agnostic explainable artificial intelligence for object detection in image data,” Eng Appl Artif Intell.
[32] Y. Li et al., “A deep learning-based hybrid framework for object detection and recognition in autonomous driving,” IEEE Access, vol. 8, pp. 194228–194239, 2020, doi: 10.1109/ACCESS.2020.3033289.
[33] M. Kuroki and T. Yamasaki, “Fast Explanation Using Shapley Value for Object Detection,” IEEE Access, vol. 12, pp. 31047–31054, 2024, doi: 10.1109/ACCESS.2024.3369890.
[34] C. Nogueira, L. Fernandes, J. N. D. Fernandes, and J. S. Cardoso, “Explaining Bounding Boxes in Deep Object Detectors Using Post Hoc Methods for Autonomous Driving Systems,” Sensors, vol. 24, no. 2, Jan. 2024, doi: 10.3390/s24020516.
[35] T. Ponn, T. Kröger, and F. Diermeyer, “Identification and explanation of challenging conditions for camera-based object detection of automated vehicles,” Sensors (Switzerland), vol. 20, no. 13, pp. 1–26, Jul. 2020, doi: 10.3390/s20133699.
[36] Tomasz Nowak, Michał R. Nowicki, Krzysztof C´wian, and Piotr Skrzypczyn´ski, “How to Improve Object Detection in a Driver Assistance System Applying Explainable Deep Learning,” IEEE, 2019.
[37] H. A. Tahir, W. Alayed, W. U. Hassan, and A. Haider, “A Novel Hybrid XAI Solution for Autonomous Vehicles: Real-Time Interpretability through LIME–SHAP Integration,” Sensors, vol. 24, no. 21, Nov. 2024, doi: 10.3390/s24216776.
[38] Mehdi Masmoudi, Hakim Ghazzai, Mounir Frikha, and Yehia Massoud, “Object Detection Learning Techniques for Autonomous Vehicle Applications,” IEEE, 2019.
[39] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection,” Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, [Online]. Available: https://goo.gl/bEs6Cj.
[40] Mohammed Bany Muhammad and Mohammed Yeasin, “Eigen-CAM: Class Activation Map using Principal Components,” IEEE, 2020.
[41] A. Chattopadhyay, A. Sarkar, P. Howlader, and V. N. Balasubramanian, “Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks,” ArXiv, Oct. 2017, doi: 10.1109/WACV.2018.00097.
[42] Glenn Jocher and Jing Qiu, “Ultralytics YOLO11,” https://github.com/ultralytics/ultralytics. Accessed: Dec. 09, 2024. [Online]. Available: https://github.com/ultralytics/ultralytics.
[43] Y. Tian, Q. Ye, and D. Doermann, “YOLOv12: Attention-Centric Real-Time Object Detectors Latency (ms) MS COCO mAP (%),” ArXiv, doi: 10.0.
[44] R. Khanam and M. Hussain, “YOLOv11: An Overview of the Key Architectural Enhancements,” Oct. 2024, [Online]. Available: http://arxiv.org/abs/2410.17725.
[45] R. Khanam and M. Hussain, “YOLOv11: An Overview of the Key Architectural Enhancements,” ArXiv, Oct. 2024, [Online]. Available: http://arxiv.org/abs/2410.17725.
[46] R. Sapkota et al., “YOLOv12 to Its Genesis: A Decadal and Comprehensive Review of The You Only Look Once (YOLO) Series,” ArXiv, Jun. 2024, [Online]. Available: http://arxiv.org/abs/2406.19407.
[47] Andreas Geiger, Philip Lenz, and Raquel Urtasun, “Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite,” IEEE, 2012.
[48] X. Li et al., “A new benchmark for vision-based cyclist detection,” in IEEE Intelligent Vehicles Symposium, Proceedings, Institute of Electrical and Electronics Engineers Inc., Aug. 2016, pp. 1028–1033. doi: 10.1109/IVS.2016.7535515.
[49] M. Cordts et al., “The Cityscapes Dataset for Semantic Urban Scene Understanding,” Apr. 2016, [Online]. Available: http://arxiv.org/abs/1604.01685.
[50] Jacob Gildenblat and contributors, “PyTorch library for CAM methods,” GitHub. Accessed: May 06, 2025. [Online]. Available: https://github.com/jacobgil/pytorch-grad-cam.
[51] z1069614715, “YOLOv11 Gradcam heatmap,” Github. Accessed: May 06, 2025. [Online]. Available: https://github.com/z1069614715/objectdetection_script/blob/master/yolo-gradcam/yolov11_heatmap.py.
[52] Z. Zou, K. Chen, Z. Shi, Y. Guo, and J. Ye, “Object Detection in 20 Years: A Survey,” May 2019, [Online]. Available: http://arxiv.org/abs/1905.05055.
[53] Q. Wang and Q. L. Wang, “BT-YOLO11: Automatic Driving Road Target Detection in Complex Scenarios,” IEEE Access, 2025, doi: 10.1109/ACCESS.2025.3562747.
[54] M. Manzoor et al., “Obstalaneyolo: Real-Time Lane and Obstacle Detection for Autonomous Vehicles,” in International Computer Conference on Wavelet Active Media Technology and Information Processing, ICCWAMTIP, Institute of Electrical and Electronics Engineers Inc., 2024. doi: 10.1109/ICCWAMTIP64812.2024.10873687.
[55] T. Clement, N. Kemmerzell, M. Abdelaal, and M. Amberg, “XAIR: A Systematic Metareview of Explainable AI (XAI) Aligned to the Software Development Process,” Mar. 01, 2023, MDPI. doi: 10.3390/make5010006.
[56] Tekedra Mawakana and Dmitri Dolgov, “Doubling down on Waymo One,” Waymo. Accessed: May 08, 2025. [Online]. Available: https://waymo.com/blog/2023/07/doubling-down-on-waymo-on.
