Advancing multi-objective reinforcement learning for natural resources management
The management and planning of natural resources are critical challenges in coping the environmental variability and anthropogenic pressures. This research proposes the development and implementation of Multi-Objective Reinforcement Learning (MORL) algorithms to address multi-objective decision-making in complex, nonlinear, stochastic, and non-stationary environmental systems. By integrating MORL techniques, the aim is to design robust and adaptive management policies that can effectively handle uncertainties inherent in real-world environmental scenarios. The study will focus on management of water reservoirs with conflicting objectives and the planning of coastal adaptation measures to mitigate risks associated with storm tides and sea-level rise. Investigating the theoretical formulation of MORL in environmental systems and its associated challenges, including the propagation of errors in Multi-Objective Fitted Q-Iteration (MOFQI) and other function approximation methods. The analysis will focus on the limitations of current MORL algorithms in handling non-stationary and convergence difficulties under high-dimensional decision spaces. ; Traditional management approaches often struggle to accommodate the multifaceted and unpredictable nature of these systems, leading to suboptimal resource utilization and increased vulnerability to environmental hazards. Reinforcement Learning (RL), offers a promising avenue for developing adaptive management strategies in such contexts. ; This research aims to develop MORL frameworks which can handle the complexities of environmental systems while ensuring adaptability to nonlinear, stochastic, and non-stationary conditions. The study involves designing state representations, reward structures, and action spaces that accurately capture the evolving nature of environmental processes. The development of reinforcement learning architectures incorporating uncertainty modeling and risk-sensitive strategies will be crucial to ensuring that resulting policies remain effective under variable and unpredictable conditions, which is essential for environmental management applications. ; A key component of this research involves conducting a theoretical and empirical analysis of MORL within environmental contexts. Special emphasis will be placed on understanding the limitations of existing MORL techniques, with a focus on function approximation and error propagation in algorithms such as MOFQI. The study will explore the compounding effects of inaccuracies in value estimation over extended learning periods and their impact on policy robustness. Empirical evaluations will assess the performance of MORL methods in multi-scale environmental optimization problems, testing their resilience under different levels of stochasticity, uncertainty, and dynamic changes. Comparative studies with traditional optimization and single-objective RL approaches will highlight the advantages and drawbacks of MORL while identifying stability issues in dynamic environments. ; To demonstrate practical applicability, MORL techniques will be applied to real-world environmental management problems. In the context of reservoir management, MORL-driven policies will be developed and tested to address trade-offs between hydropower generation, flood control, and ecological flow maintenance, with an evaluation of their adaptability to changing climatic conditions, hydrological variability, and competing stakeholder demands. In coastal adaptation planning, MORL methods will be used to optimize the selection and timing of adaptation measures for mitigating storm tide risks and addressing long-term sea-level rise. Probabilistic climate projections will be incorporated, and different adaptation strategies will be evaluated under various environmental and socio-economic scenarios. ; This research is expected to contribute to the field of environmental decision-making by extending the application of RL to complex environmental systems, demonstrating its potential in developing adaptive and robust management strategies. Additionally, it seeks to advance MORL methodologies tailored to real-world environmental constraints and uncertainties. A key aspect of this research is providing an empirical analysis of MORL algorithm weaknesses, particularly regarding function approximation and convergence stability. Finally, this work will demonstrate the applicability of MORL in complex multi-objective environmental management problems, paving the way for more resilient and adaptive decision-making frameworks.
Back to Current Students