Explainable Artificial Intelligence Methods for Autonomous Robot Decision Making: A Multi Agent Framework with Safety Assurance and Ethical Constraint Optimization

Harry Setya Hadi; Nicodemus Rahanra

Authors

Harry Setya Hadi Universitas Ekasakti
Nicodemus Rahanra Universitas Satya Wiyata Mandala

Keywords:

Autonomous decision making, Explainable artificial intelligence, Safety-critical systems, Transparent AI, Trustworthy AI

Abstract

Autonomous decision-making systems increasingly rely on complex artificial intelligence models to operate in dynamic and safety-critical environments. While these models provide strong predictive capabilities, their black-box nature limits transparency, trust, and accountability. This study proposes a structured research methodology for integrating Explainable Artificial Intelligence (XAI) into autonomous decision-making systems. The research adopts a conceptual–analytical approach to develop an explainability-oriented framework that embeds transparency across perception, decision-making, and action execution stages. The methodology includes literature-driven problem identification, conceptual framework construction, classification and mapping of XAI methods, and formulation of explainability evaluation criteria. The results demonstrate that effective explainability in autonomous systems requires a hybrid integration strategy, combining in-model transparency with post-hoc explanation mechanisms. A structured mapping of XAI techniques to autonomous system components and a conceptual decision-flow diagram are presented to illustrate explainability integration. The findings highlight that layered and context-aware explainability enhances system interpretability, supports human oversight, and improves safety relevance without compromising autonomous operation. This study contributes a reusable methodological foundation for the design and evaluation of explainable autonomous systems, offering practical guidance for future empirical validation and real-world deployment in safety-critical applications.

References

[1] S. Hemalatha, K. V. S. V. T. Reddy, T. V Rao, T. Ramaswamy, N. M. Pillai, and G. K. Mohan, “Advancing autonomous systems: A review of emerging trends in robotics,” J. Eur. des Systèmes Autom., vol. 58, no. 5, pp. 913–921, 2025, doi: 10.18280/jesa.580505.

[2] C. Thames and Y. Sun, “A survey of artificial intelligence approaches to safety and mission-critical systems,” in Integrated Communications, Navigation and Surveillance Conference, 2024. doi: 10.1109/ICNS60906.2024.10550712.

[3] A. Sevgi Ostim, A. M. Kadiouglu Ostim, and A. Durmucs Ostim, “Using of robotic systems in transportation,” in Lecture Notes in Networks and Systems, Springer, 2025, pp. 458–468. doi: 10.1007/978-3-031-81799-1_42.

[4] U. A. Usmani, A. Happonen, and J. Watada, “Revolutionizing transportation: Advancements in robot-assisted mobility systems,” in Lecture Notes in Networks and Systems, Springer, 2023, pp. 603–619. doi: 10.1007/978-981-99-4932-8_55.

[5] S. Akhai, M. Abbass, P. Kaur, and T. Kaur, “Digital transformation across generations: Robotics and AI in action,” in Impacts of Digital Technologies Across Generations, IGI Global, 2025, pp. 23–39. doi: 10.4018/979-8-3693-6366-9.ch002.

[6] K. Thamrongaphichartkul, N. Worrasittichai, T. Prayongrak, and S. Vongbunyong, “Development of autonomous mobile service robot with safety standard for cart moving applications in hospitals,” in AIP Conference Proceedings, 2024, p. 70006. doi: 10.1063/5.0205057.

[7] S. Katzenbeisser, I. Polian, F. Regazzoni, and M. Stottinger, “Security in autonomous systems,” in Proceedings of the European Test Workshop, 2019. doi: 10.1109/ETS.2019.8791552.

[8] M. Hamad and S. Steinhorst, “Security challenges in autonomous systems design,” in Communications in Computer and Information Science, Springer, 2025, pp. 142–154. doi: 10.1007/978-3-031-81981-0_13.

[9] E. Şahin, N. N. Arslan, and D. Özdemir, “Unlocking the black box: An in-depth review on interpretability, explainability, and reliability in deep learning,” Neural Comput. Appl., vol. 37, no. 2, pp. 859–965, 2025, doi: 10.1007/s00521-024-10437-2.

[10] V. Buhrmester, D. Münch, and M. Arens, “Analysis of explainers of black box deep neural networks for computer vision: A survey,” Mach. Learn. Knowl. Extr., vol. 3, no. 4, pp. 966–989, 2021, doi: 10.3390/make3040048.

[11] F. Emmert-Streib, O. Yli-Harja, and M. Dehmer, “Explainable artificial intelligence and machine learning: A reality rooted perspective,” Wiley Interdiscip. Rev. Data Min. Knowl. Discov., vol. 10, no. 6, p. e1368, 2020, doi: 10.1002/widm.1368.

[12] Q. Teng, Z. Liu, Y. Song, K. Han, and Y. Lu, “A survey on the interpretability of deep learning in medical diagnosis,” Multimed. Syst., vol. 28, no. 6, pp. 2335–2355, 2022, doi: 10.1007/s00530-022-00960-4.

[13] X. Cheng et al., “Explainability in GeoAI,” in Handbook of Geospatial Artificial Intelligence, CRC Press, 2023, pp. 177–200. doi: 10.1201/9781003308423-9.

[14] S. B. Kasetty and K. Rajakumar, “Understanding satellite image processing and black box models with ante-hoc and post-hoc explanations in deep learning,” in Proceedings of the 2024 10th International Conference on Communication and Signal Processing (ICCSP), 2024, pp. 848–853. doi: 10.1109/ICCSP60870.2024.10544196.

[15] Z. M. Arkah, B. Pontes, and C. Rubio, “Interpretation of Diabetic Foot Ulcer Image Classification Using Layer Attribution Algorithms,” in Lecture Notes in Networks and Systems, 2024, pp. 13–22. doi: 10.1007/978-3-031-75013-7_2.

[16] D. S. Sujana and D. P. Augustine, “Explaining autism diagnosis model through local interpretability techniques—A post-hoc approach,” in 2023 International Conference on Data Science, Agents and Artificial Intelligence (ICDSAAI), 2023. doi: 10.1109/ICDSAAI59313.2023.10452575.

[17] C. Ren, Y. Xu, and R. Zhang, “An interpretable deep learning method for power system transient stability assessment via tree regularization,” IEEE Trans. Power Syst., vol. 37, no. 5, pp. 3359–3369, 2022, doi: 10.1109/TPWRS.2021.3133611.

[18] J. García-Sigüenza, F. Llorens-Largo, L. Tortosa, and J. F. Vicent, “Explainability techniques applied to road traffic forecasting using graph neural network models,” Inf. Sci. (Ny)., vol. 645, p. 119320, 2023, doi: 10.1016/j.ins.2023.119320.

[19] B. Oliveira and C. C. Leal, “AI in finance: Applications and challenges,” in Challenges and Opportunities in the Artificial Intelligence Era, Springer, 2025, pp. 79–107. doi: 10.1007/978-3-031-85272-5_6.

[20] A. Mishra and M. Malhotra, “A dual approach with Grad-CAM and layer-wise relevance propagation for CNN models explainability,” in Communications in Computer and Information Science, Springer, 2025, pp. 116–129. doi: 10.1007/978-3-031-80842-5_10.

[21] K. Bzhikhatlov, O. Nagoeva, M. Anchokov, and D. Makoeva, “Methods and algorithms (modeling of reasoning) to synthesize intellectual behavior of autonomous mobile robots and program complexes based on received reasoning models,” in Studies in Computational Intelligence, vol. 477, Springer, 2024, pp. 87–98. doi: 10.1007/978-3-031-76516-2_7.

[22] Y. Zhou and W. Zhang, “End-to-end robot intelligent obstacle avoidance method based on deep reinforcement learning with spatiotemporal transformer architecture,” Front. Neurorobot., vol. 19, p. 1646336, 2025, doi: 10.3389/fnbot.2025.1646336.

[23] H. Bavle, J. L. Sanchez-Lopez, C. Cimarelli, A. Tourani, and H. Voos, “From SLAM to situational awareness: Challenges and survey,” Sensors, vol. 23, no. 10, p. 4849, 2023, doi: 10.3390/s23104849.

[24] B. Abdelkader, N. Emira, and E. Nadjib, “From perception to action: Transformer-enhanced deep reinforcement learning for autonomous robot navigation,” in Proceedings of the 7th International Conference on Pattern Analysis and Intelligent Systems (PAIS), 2025. doi: 10.1109/PAIS66004.2025.11126486.

[25] S. Lokhande, J. Dailey, Y. Liu, S. Connolly, and H. Xu, “A novel explainable AI based situation recognition for autonomous robots with partial unlabeled data,” in Proceedings of SPIE, 2023, p. 1254606. doi: 10.1117/12.2664016.

[26] M. Mersha, K. Lam, J. Wood, A. K. AlShami, and J. Kalita, “Explainable artificial intelligence: A survey of needs, techniques, applications, and future direction,” Neurocomputing, vol. 599, p. 128111, 2024, doi: 10.1016/j.neucom.2024.128111.

[27] T. H. Sardar, S. Das, and B. K. Pandey, “Explainable AI (XAI): Concepts and theory,” in Medical Data Analysis and Processing using Explainable Artificial Intelligence, CRC Press, 2023, pp. 1–18. doi: 10.1201/9781003257721-1.

[28] M. H. Azam, M. H. Hasan, N. Y. Murad, and E. A. B. Patah, “Transparency in AI: A review of explainable artificial intelligence techniques,” in 2024 8th International Conference on Computing, Communication, Control and Automation (ICCUBEA), 2024. doi: 10.1109/ICCUBEA61740.2024.10774981.

[29] B. Mohammed, “A review on explainable artificial intelligence methods, applications, and challenges,” Indones. J. Electr. Eng. Informatics, vol. 11, no. 4, pp. 1007–1024, 2023, doi: 10.52549/ijeei.v11i4.5151.

[30] G. M. Alarcon and S. M. Willis, “Explaining explainable artificial intelligence: An integrative model of objective and subjective influences on XAI,” in Proceedings of the Annual Hawaii International Conference on System Sciences, 2023, pp. 1095–1104.

[31] S. Bhatnagar and R. Agrawal, “Understanding explainable artificial intelligence techniques: A comparative analysis for practical application,” Bull. Electr. Eng. Informatics, vol. 13, no. 6, pp. 4451–4455, 2024, doi: 10.11591/eei.v13i6.8378.

[32] A. Singhal, P. Pratap, K. K. Dixit, and K. Kathuria, “Advancements in explainable AI: Bridging the gap between model complexity and interpretability,” in 2024 2nd International Conference on Disruptive Technologies (ICDT), 2024, pp. 675–680. doi: 10.1109/ICDT61202.2024.10489277.

[33] Z. Cheng, Y. Wu, Y. Li, L. Cai, and B. Ihnaini, “A comprehensive review of explainable artificial intelligence (XAI) in computer vision,” Sensors, vol. 25, no. 13, p. 4166, 2025, doi: 10.3390/s25134166.

[34] B. P. Sheela and H. Girisha, “An explainable artificial intelligence (XAI) framework for deep learning based classification to generate textual explanations on predicted images,” Int. J. Intell. Eng. Syst., vol. 17, no. 6, pp. 651–662, 2024, doi: 10.22266/ijies2024.1231.50.

[35] M. Thamer and Z. N. Sultani, “Explainable AI in the medical field: A survey on machine learning interpretability and use cases,” Al-Nahrain J. Sci., vol. 28, no. 4, pp. 188–206, 2025, doi: 10.22401/ANJS.28.4.15.

[36] A. Kuznietsov, B. Gyevnar, C. Wang, S. Peters, and S. V Albrecht, “Explainable AI for safe and trustworthy autonomous driving: A systematic review,” IEEE Trans. Intell. Transp. Syst., vol. 25, no. 12, pp. 19342–19364, 2024, doi: 10.1109/TITS.2024.3474469.

[37] R. K. Shinde, K. D. Shinde, and H. Mehta, “A hybrid explainable AI framework for enhancing trust and transparency in autonomous vehicles,” in 2025 International Conference on Emerging Smart Computing and Informatics (ESCI), 2025. doi: 10.1109/ESCI63694.2025.10987990.

[38] K. Malik, M. Sharma, S. Deswal, U. Gupta, D. Agarwal, and Y. O. B. Al Shamsi, Explainable artificial intelligence for autonomous vehicles: Concepts, challenges, and applications. CRC Press, 2024. doi: 10.1201/9781003502432.

[39] S. Javaid, M. A. Khan, H. Fahim, B. He, and N. Saeed, “Explainable AI and monocular vision for enhanced UAV navigation in smart cities: Prospects and challenges,” Front. Sustain. Cities, vol. 7, p. 1561404, 2025, doi: 10.3389/frsc.2025.1561404.

[40] S. S. Malwade and S. J. Budhavale, “Exploring explainable AI: Current trends, challenges, techniques and its applications,” in ACM International Conference Proceeding Series, 2023, p. 85. doi: 10.1145/3647444.3647912.

[41] M. Kopzhasarova and D. Kozhamzharova, “Explainable AI (XAI): Techniques, applications, and challenges,” in CEUR Workshop Proceedings, 2025.

[42] A. Apicella, L. Di Lorenzo, F. Isgrò, A. Pollastro, and R. Prevete, “Strategies to exploit XAI to improve classification systems,” in Communications in Computer and Information Science, Springer, 2023, pp. 147–159. doi: 10.1007/978-3-031-44064-9_9.

[43] G. Türkmen, “The Review of Studies on Explainable Artificial Intelligence in Educational Research,” J. Educ. Comput. Res., vol. 63, no. 2, pp. 277–310, 2025, doi: 10.1177/07356331241310915.