Design and Implementation of a Deep Reinforcement Learning Framework for Autonomous Navigation in Dynamic Unstructured Robotic Environments with Real Time Obstacle Avoidance

Milli Alfhi Syari; Zira Fatmaira; Syofyan Anwar syahputra

Authors

Milli Alfhi Syari STMIK Kaputama
Zira Fatmaira STMIK Kaputama
Syofyan Anwar syahputra Akademi Teknik Deli Serdang

Keywords:

Autonomous Robot Navigation, Dynamic Environments, Obstacle Avoidance, Path Planning, Conventional Navigation Algorithms

Abstract

Autonomous robot navigation in dynamic and unstructured environments remains a critical challenge due to unpredictable obstacles, sensor uncertainty, and limited adaptability of traditional planning algorithms. Although conventional navigation methods such as graph-based, potential field–based, and sampling-based approaches have been widely adopted, their performance under real-time dynamic conditions is still constrained. This study aims to design and implement a comprehensive experimental framework to evaluate the effectiveness and limitations of conventional navigation algorithms for autonomous mobile robots operating in dynamic unstructured environments. The research adopts an experimental and comparative methodology by implementing A*, Dijkstra, Artificial Potential Field (APF), and Rapidly-Exploring Random Tree (RRT) algorithms in simulated static and dynamic scenarios. Performance is assessed using quantitative metrics including path length, computation time, success rate, collision rate, and path smoothness. The experimental results demonstrate that graph-based algorithms achieve high success rates and optimal path efficiency in static environments but exhibit limited adaptability to dynamic changes. APF offers fast computation but suffers from high collision rates due to local minima, while RRT shows better adaptability in dynamic environments at the cost of longer and less smooth paths. These findings confirm that conventional navigation methods are insufficient for robust autonomous navigation in highly dynamic and unstructured environments. The study highlights the necessity of adaptive and learning-based navigation frameworks, such as deep reinforcement learning, to enhance real-time decision-making, robustness, and autonomy in future robotic systems.

References

[1] N. A. K. Zghair and A. S. Al-Araji, “A one decade survey of autonomous mobile robot systems,” Int. J. Electr. Comput. Eng., vol. 11, no. 6, pp. 4891–4906, 2021, doi: 10.11591/ijece.v11i6.pp4891-4906.

[2] N. Gogoi, A. Minetto, and F. Dovis, “A proof-of-concept of cooperative DGNSS for UAV/UGV navigation,” in Proceedings of the 33rd International Technical Meeting of the Satellite Division of the Institute of Navigation (ION GNSS+ 2020), 2020, pp. 2229–2236. doi: 10.33012/2020.17529.

[3] L. N. Patil et al., “Precision mapping and navigation: A robotic restaurant management system using SLAM and ROS,” Sigma J. Eng. Nat. Sci., vol. 43, no. 4, pp. 1233–1247, 2025, doi: 10.14744/sigma.2025.00117.

[4] Y. Tao, H. Liu, T. Wang, D. Han, and G. Zhao, “Research progress and industrialization development trend of Chinese service robot,” J. Mech. Eng., vol. 58, no. 18, pp. 56–74, 2022, doi: 10.3901/JME.2022.18.056.

[5] A. Williams, B. Sebastian, and P. Ben-Tzvi, “Review and analysis of search, extraction, evacuation, and medical field treatment robots,” J. Intell. Robot. Syst., vol. 96, no. 3--4, pp. 401–418, 2019, doi: 10.1007/s10846-019-00991-6.

[6] S. M. Nasti and M. A. Chishti, “A review of AI-enhanced navigation strategies for mobile robots in dynamic environments,” in Proceedings of the ASU International Conference in Emerging Technologies for Sustainability and Intelligent Systems (ICETSIS 2024), 2024, pp. 1239–1244. doi: 10.1109/ICETSIS61505.2024.10459613.

[7] D. Singh, A. Singh, S. K. Joshi, and R. Deorari, “Obstacle avoidance, drive wheel synchronization, and line tracking capabilities in an autonomous robot,” in Proceedings of the International Conference on Advances in Computing, Communication and Materials (ICACCM 2022), 2022. doi: 10.1109/ICACCM56405.2022.10009273.

[8] S. R. R. Vadivel, R. K. Megalingam, A. P. P. Sreelatha, and A. P. Udayan, “Enhancing indoor mobility: A novel suspended drive wheel mechanism for differential drive mobile robots,” in Proceedings of the 4th Asian Conference on Innovation in Technology (ASIANCON 2024), 2024. doi: 10.1109/ASIANCON62057.2024.10837746.

[9] J. Knoop and D. Schreiner, “Software aspects of robotic systems,” in Lecture Notes in Computer Science, vol. 7610, Springer, 2012, p. 323. doi: 10.1007/978-3-642-34032-1_30.

[10] L. Wijayathunga, A. Rassau, and D. Chai, “Challenges and solutions for autonomous ground robot scene understanding and navigation in unstructured outdoor environments: A review,” Appl. Sci., vol. 13, no. 17, p. 9877, 2023, doi: 10.3390/app13179877.

[11] M. A. Taleb, G. Korsoveczki, and G. Husi, “Automotive navigation for mobile robots: Comprehensive review,” Results Eng., vol. 27, p. 105837, 2025, doi: 10.1016/j.rineng.2025.105837.

[12] C. Mavrogiannis, “Towards smooth mobile robot deployments in dynamic human environments,” AI Mag., vol. 45, no. 3, pp. 419–428, 2024, doi: 10.1002/aaai.12192.

[13] H. Zhou, P. Feng, and W. Chou, “A hybrid obstacle avoidance method for mobile robot navigation in unstructured environment,” Ind. Rob., vol. 50, no. 1, pp. 94–106, 2023, doi: 10.1108/IR-04-2022-0102.

[14] M. Z. Butt, N. Nasir, R. B. A. Rashid, and A. M. Z. B. A. Ahmad, “Virtual-target-based reactive and non-cooperative obstacle avoidance: Application in low-altitude autonomous aerial navigation in outdoor unstructured environments,” Eng. Res. Express, vol. 7, no. 4, p. 45537, 2025, doi: 10.1088/2631-8695/ae1283.

[15] M. Skoczeń et al., “Obstacle detection system for agricultural mobile robot application using RGB-D cameras,” Sensors, vol. 21, no. 16, p. 5292, 2021, doi: 10.3390/s21165292.

[16] W. Sun, D. Shen, K. Liu, Z. Li, and B. Zhou, “Multi-sensor fusion-based perception and navigation framework for quadruped robots in complex environments,” in Chinese Control Conference (CCC), 2024, pp. 4561–4566. doi: 10.23919/CCC63176.2024.10662127.

[17] T. Zhang, H. Zhang, and X. Li, “Vision-audio fusion SLAM in dynamic environments,” CAAI Trans. Intell. Technol., vol. 8, no. 4, pp. 1364–1373, 2023, doi: 10.1049/cit2.12206.

[18] Y. Zhang et al., “Online efficient safety-critical control for mobile robots in unknown dynamic multi-obstacle environments,” in IEEE International Conference on Intelligent Robots and Systems (IROS), 2024, pp. 12370–12377. doi: 10.1109/IROS58592.2024.10802727.

[19] M. Basavanna, M. Shivakumar, and K. R. Prakash, “Navigation of mobile robot through mapping using Orbbec Astra camera and ROS in an indoor environment,” in Lecture Notes in Mechanical Engineering, 2022, pp. 465–474. doi: 10.1007/978-981-16-4222-7_53.

[20] D. D. J. García Jiménez, T. Olvera, U. Orozco-Rosas, and K. Picos, “Autonomous object manipulation and transportation using a mobile service robot equipped with an RGB-D and LiDAR sensor,” in Proceedings of SPIE, 2021, p. 118410K. doi: 10.1117/12.2594025.

[21] Z. Wei, S. Wang, K. Chen, and F. Wang, “ROS-based navigation and obstacle avoidance: A study of architectures, methods, and trends,” Sensors, vol. 25, no. 14, p. 4306, 2025, doi: 10.3390/s25144306.

[22] J. Zhang, S. Wang, Y. Zhang, and J. Zhou, “Autonomous obstacle avoidance motion planning for mobile robots in dark and weak environments based on multimodal information fusion,” in Journal of Physics: Conference Series, 2025, p. 12003. doi: 10.1088/1742-6596/3077/1/012003.

[23] S. Harapanahalli, N. O. Mahony, G. V Hernandez, S. Campbell, D. Riordan, and J. Walsh, “Autonomous navigation of mobile robots in factory environment,” Procedia Manuf., vol. 38, pp. 1524–1531, 2019, doi: 10.1016/j.promfg.2020.01.134.

[24] R. W. S. M. de Oliveira et al., “A robot architecture for outdoor competitions,” J. Intell. Robot. Syst., vol. 99, no. 3--4, pp. 629–646, 2020, doi: 10.1007/s10846-019-01140-9.

[25] B. Khanal, A. Happonen, J. Heikkonen, and R. Kanth, “Autonomous quadruped robot system with LiDAR sensor navigation and task execution,” in Proceedings of the 14th Mediterranean Conference on Embedded Computing (MECO 2025), 2025. doi: 10.1109/MECO66322.2025.11049177.

[26] A. Mellouk and A. Benmachiche, “A survey on navigation systems in dynamic environments,” in ACM International Conference Proceeding Series, 2020. doi: 10.1145/3447568.3448527.

[27] H. Q. T. Ngo, “Recent researches on human-aware navigation for autonomous system in the dynamic environment: An international survey,” in Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, vol. 409, 2021, pp. 267–282. doi: 10.1007/978-3-030-93179-7_21.

[28] S. Sethi, “Path planning algorithms in robotics,” in Proceedings of the 2025 International Conference on Computational Innovations and Engineering Sustainability (ICCIES 2025), 2025. doi: 10.1109/ICCIES63851.2025.11032663.

[29] C. B. Kolanur, J. Bali, S. Tanvashi, and A. Giriyapur, “An overview of collision-free path planning techniques for industrial autonomous robots,” in Cyber-Physical Systems: Applications, Challenges, and Research Directions, 2025, pp. 1–17. doi: 10.1201/9781779643421-1.

[30] M. D. Y. Kumar and K. Rajchandar, “Bi-directional virtual search algorithm for efficient and collision-free path planning in autonomous robots navigating static and dynamic environments,” Ain Shams Eng. J., vol. 16, no. 9, p. 103526, 2025, doi: 10.1016/j.asej.2025.103526.

[31] A. Lazarowska, “Discrete artificial potential field approach to mobile robot path planning,” IFAC-PapersOnLine, vol. 52, no. 8, pp. 334–337, 2019, doi: 10.1016/j.ifacol.2019.08.083.

[32] Q. Wu et al., “Real-time dynamic path planning of mobile robots: A novel hybrid heuristic optimization algorithm,” Sensors, vol. 20, no. 1, p. 188, 2020, doi: 10.3390/s20010188.

[33] A. Singh, Y. Sridhar, V. Kalaichelvi, and R. Karthikeyan, “Performance evaluation of vision based path planning for dynamic real-time scenarios of mobile robot,” Multimed. Tools Appl., vol. 84, no. 10, pp. 7377–7400, 2025, doi: 10.1007/s11042-024-19267-9.

[34] D. Danang, H. Haryani, Q. Aini, F. A. Ramahdan, and J. Edwards, “Empowering digital literacy through blockchain based alphasign for secure and sustainable e-governance,” 2025.

[35] D. Danang, A. B. Santoso, and M. U. Dewi, “CICA Framework: Harnessing CSR, AI, and Blockchain for Sustainable Digital Culture,” Int. J. Adv. Comput. Sci. & Appl., vol. 16, no. 11, 2025.

[36] D. Danang, T. Wahyono, I. Sembiring, T. Wellem, and N. H. Dzulkefly, “An Adaptive Framework Integrating ML Blockchain and TEE for Cloud Security,” in 2025 4th International Conference on Creative Communication and Innovative Technology (ICCIT), 2025, pp. 1–7.

[37] D. Danang, E. Siswanto, N. D. Setiawan, and P. Wibowo, “Hybrid Zero Trust Container Based Model for Proactive Service Continuity under Intelligent DDoS Attacks in Cloud Environment,” Int. J. Comput. Technol. Sci., vol. 2, no. 3, pp. 41–49, 2025, doi: 10.62951/ijcts.v2i3.291.

[38] D. Danang, M. U. Dewi, and G. Widhiati, “Federated Hybrid CNN GRU and COBCO Optimized Elman Neural Network for Real Time DDoS Detection in Cloud Edge Environments,” Int. J. Electr. Eng. Math. Comput. Sci., vol. 2, no. 2, pp. 28–35, 2025, doi: 10.62951/ijeemcs.v2i2.293.

[39] H. R. D. Putranti, D. Danang, T. Da Silva, and A. A. B. Pujiati, “Integrating Hands-on and Virtual Learning for Environmental Sustainability: Eco Enzyme Soap Making at Stella Matutina,” REKA ELKOMIKA J. Pengabdi. Kpd. Masy., vol. 6, no. 1, pp. 88–97, 2025.