Pusat Pengajian Kejuruteraan Aeroangkasa - Tesis

Browse

Browsing Pusat Pengajian Kejuruteraan Aeroangkasa - Tesis by Author "Feng, Yiting"

Now showing 1 - 1 of 1
  • Thumbnail Image
    Publication
    Online reinforcement learning with multivariate splines for flight control.
    (2025-12)
    Feng, Yiting
    In recent decades, Unmanned Aircraft Vehicle (UAV)s have been increasingly deployed across military, industrial, agricultural, and rescue missions, placing higher demands on autonomous flight control. Reinforcement Learning (RL) has emerged as a promising approach due to its self-learning capability, and its combination with dynamic programming has led to Adaptive Dynamic Programming (ADP), which offers both optimality and adaptability. However, applying RL/ADP to flight control is still challenging, as large Artificial Neural Networks (ANNs) increase computational load and hinder stability analysis, the lack of a unified ADP framework reduces online learning efficiency, and uncertainties, disturbances, and system constraints further complicate controller design. In addition, purely data-driven RL makes it difficult to interpret control performance from the system dynamics. To address these issues, this thesis develops a series of RL-based control methods enhanced by multivariate spline techniques. Firstly, multivariate B-splines are incorporated into the RL approximation frame work to enhance learning efficiency while maintaining a comparable number of param eters to Neural Network (NN). Quantitative evaluations demonstrate that the proposed spline-based RLapproach yieldsfasterconvergenceofsystemstatesandachieveshigher accumulated rewards. Compared with the NN-based method, the cumulative control energy isreducedby28.4%,andthetrainingtimeaccountsforonly9.7%ofthatrequired by the NNbaseline. The convergence of the value functions indicates clear differences between the two approaches: the NN-based controller exhibits continuously increas ing energy consumption, showing that its control moment fails to converge, whereas the spline-based method maintains stable energy behavior. After that, an incremental model-based Adaptive Critic Designs (ACDs) based on tensor-product B-splines is developed to improve online learning stability. Under model uncertainties and external disturbances applied to the helicopter dynamics, system states remain close to their desired values with only small variations, and the initial tracking errors decay rapidly andstay near zero for bothcontrollers, with the spline-based method exhibiting stronger disturbance rejection. Then, a novel robust adaptive optimal control paradigm is further proposed with explicit consideration of both input saturation and state constraints. A concurrent learn ing mechanism is incorporated into the actor-critic structure to relax the Persistence of Excitation (PE) condition. Through applications to attitude and altitude tracking of an unmanned helicopter, the proposed controller demonstrates the capability to handle constrained inputs and outputs while maintaining stable adaptive optimal performance. Finally, for full-loop quadrotor trajectory tracking, the contribution of individual controller components is analyzed through quantitative comparisons. Removing the Nonlinear Disturbance Observer (NDO) leads to noticeable degradation, where position trajectories fail to converge and the settling time becomes unbounded. With out the Prescribed Performance Control (PPC) module, the controller cannot explicitly regulate the error dynamics, resulting in reduced tracking precision and diminished stability margins. In contrast, the complete proposed controller maintains the tracking error within the desired bounds throughout the flight and ensures fast response and high accuracy under both nominal and disturbed conditions.