Publication: Reinforcement learning-integrated hierarchical multi-agent Framework for cooperative indoor exploration.
No Thumbnail Available
Date
2026-10
Authors
Low, Derrick
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Autonomous exploration of unknown indoor corridor networks by nano-quadrotors is challenging because achieving verifiable complete branch coverage requires no metric map and no external localisation, under the sensing limits of a four-directional Time-of-Flight and optical-flow sensor suite matched to the Crazyflie CF2.X platform. Existing frontier-based methods require an occupancy grid that degrades under odometric drift, while micromouse algorithms provide complete coverage guarantees but assume discrete grid representations and binary wall sensors unavailable on continuous-environment platforms. A further gap exists at the multi-agent level: when multiple agents explore the same corridor network cooperatively, peer body artefacts corrupt junction classification, and standard branch-memory approaches do not partition exploration work or prevent concurrent branch entry without a shared metric map.This thesis addresses both gaps with a four-layer hierarchical architecture that combines a learned, width-invariant junction classifier with a generation-based topological
memory and a deterministic rule-based coordination layer, extended to cooperative multi-agent operation without joint training or centralised control, and evaluates both single-agent and multi-agent configurations in simulation. The system was implemented in PyBullet using hardware-matched Crazyflie CF2.X parameters; all results are obtained in simulation, with hardware parameter matching intended to support, not substitute for, future physical validation. A Proximal Policy Optimisation classifier was trained via a four-phase curriculum to identify fourteen discrete junction types from a twelve-dimensional observation vector of width-normalised range readings and onboard heading, with no position information supplied. A generation-based topological memory records per-branch exploration status for every confirmed junction,and a depth-first search coordinator provides a formal coverage guarantee that every reachable branch is entered exactly once in the forward direction. The multi-agent extension requires no modification to the learned junction classifier: it adds a branch status ownership protocol via a shared junction database, an explorer and supporter role-assignment framework, and peer body masking, enabling cooperative exploration as a Decentralised Partially Observable Markov Decision Process with Communication. Four independent out-of-distribution sweeps spanning corridor width, branch length, and heading noise gave 77.4% pooled accuracy, with the sharpest degradation driven by branches shorter than the corridor is wide rather than by corridor width itself. The single-agent system achieved 100% coverage completeness across three maze grid sizes and a 2× corridor width range (0.6–1.2 m) without retraining. Multi-agent exper iments scaling from one to four agents achieved at least 99% mean coverage at every agent count, with scaling efficiency falling from 94–97% at two agents to 65.7–72.1% at four agents as coordination overhead grows relative to the shrinking exploration territory assigned to each agent, confirming that the branch-status protocol achieves complete work partitioning without joint training or centralised coordination even as agent count increases. These results indicate that topological memory with position proximity matching, paired with a minimally learned perception module, is a practical
route to infrastructure-free indoor exploration on sensing-constrained nano-quadrotor platforms, with direct application to search-and-rescue and infrastructure-inspection settings where metric maps and external localisation are unavailable.