Publication: Hierarchical deep deterministic policy gradient algorithm for mobile robot navigation.
Loading...
Date
2025-10
Authors
Hu, Wenjie
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Autonomous navigation for mobile robots in both static and dynamic environments presents significant challenges, necessitating robust and efficient algorithms. Traditional approaches, such as simultaneous localization and mapping (SLAM), often rely on computationally demanding global and local maps, leading to slow convergence and extended training times. Deep reinforcement learning (DRL) has emerged as a promising alternative, yet existing methods face limitations, including inefficient exploration, overestimation issues, and difficultly handling long-horizon planning in complex maze-like environments. To address these challenges, this dissertation proposes three advanced DRL-based algorithms for autonomous mobile robot navigation. The first of these, the prioritized experience replay noisy n-step dueling double deep 𝑄-Network (PER-n2D3QN) algorithm, is specifically designed for discrete action spaces. Empirical evaluations demonstrate that PER-n2D3QN substantially outperforms conventional approaches in terms of accuracy, convergence speed, and robustness across diverse navigation tasks. To extend to a continuous action space, the adaptive parameter space noise double critics and double actors deep deterministic policy gradient (AP-D4PG) algorithm is introduced. Multiple evaluations show that AP-D4PG enhances the success rate by a minimum of approximately 3.84% and increases the average reward by at least roughly 48.93% when compared to the benchmark algorithm. Furthermore, the hierarchical deep deterministic policy gradient (HDDPG) algorithm is developed for autonomous maze navigation, tackling challenges associated with sparse rewards, inefficient exploration, and long-horizon planning. Empirical assessment in maze navigation tasks demonstrate that HDDPG significantly improves navigation efficiency, increasing the average success rate by more than 70%, and the average reward by more than 568. By refining network architectures, exploration strategies, and reward mechanisms, this dissertation advancesthestateofautonomousnavigationformobilerobots. Through extensive simulation experiments in ROS and Gazebo, this research validates the effectiveness and performance of the proposed methods to achieve effective, robust, and adaptive autonomous navigation for mobile robots in both static and dynamic environments. This study not only advances the theoretical foundations of DRL and HDRL, but also provides a highly valuable reference through comprehensive simulation experiments for practically developing intelligent mobile robots capable of operating in autonomous navigation.