Publication: Automated reward shaping with domain knowledge for reinforcement learning-based path planning
Date
2024-08-01
Authors
Tan, Chee Sheng
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Researchers have leveraged existing knowledge within the field of reinforcement learning (RL) to address path planning problems effectively. A heuristic approach or target information can be used to reduce the search space, but they require extra information. Exploring alternative approaches, such as shifting focus to the Q-table, provides another solution to path planning problems. In light of this, a novel expected-mean gamma-incremental Q approach that integrates domain knowledge from the Q-table for more successful problem-solving outcomes, ultimately achieving optimal path planning. This study proposes an adaptive reward shaping-based RL framework for addressing multi-objective path planning problems, which incorporates key criteria such as minimizing path cost, ensuring safety, achieving smooth routes, and preventing local minima through meaningful reward functions. To address this, a new algorithm, the improved expected-mean gamma-incremental Q (post-optimization), has been developed by integrating an enhanced updating rule and optimized reward shaping weights. The results from simulations clearly demonstrate the effectiveness of the suggested approach to path planning, utilizing the proposed algorithm. It has been shown to outperform the traditional Q-learning by achieving convergence up to twice as fast. Also, this approach displays superior resilience in achieving multiple objectives simultaneously, as evidenced by achieving the highest total average return of -247.05 in solving path planning problems. In conclusion, the outcome from this study has the potential to solve the path planning problem using expert system and can provide significant contribution in the robotic application environment.