Uncovering the Solution Space of Chaotic Systems via Multi-Objective Reinforcement Learning

Missions operating in chaotic, multi-body environments have long been at the forefront of NASA’s goal to better humanity’s understanding of the solar system. Systems such as the Jovian, Saturnian, and Pluto-Charon systems have hosted some of the most significant missions of recent years with compelling, critical science being performed that have illuminated the scientific community with numerous insights. Due to the complex nature of these missions, the trajectory design process becomes an integral part of these missions that serves to enable fuel mass savings, thorough exploration within the system, and mission feasibility. However, trajectory design for these missions has long been inhibited by the high dimensionality that is inherent to the trajectory design process. This high dimensionality prohibits a thorough exploration of the design space, and necessarily constrains the design process to only examining a limited number of mission architectures. One technique to explore multi-body systems is to utilize the Circular Restricted Three-Body Problem (CR3BP), but scenarios still exist where no clear initial guess can be formed for a mission necessitating the development of new techniques for concept development, mission extensions, and in-orbit redesign. To enhance the design space exploration using the CR3BP and uncover solutions that would further enable missions in multi-body gravitational environments, advancements in machine learning, namely Multi-Objective Reinforcement Learning (MORL), can be utilized to explore the global trajectory design space. However, directly applying machine learning techniques in a dynamical system can be computationally expensive leading to the need to constructively bias an algorithm with dynamical systems theory insights in order to speed up convergence and uncover a more diverse solution space. To achieve this goal, the following research objectives will be completed: 1) Define parameters of the MORL algorithm in a chaotic, dynamical model, 2) Incorporate solution constraints and insights from dynamical systems theory, 3) Implement the algorithm within a natural and a low-thrust enabled CR3BP, 4) Characterize the performance of the methodology and assess the capability for use in mission design and higher fidelity models. This research project addresses the objectives specified in the 2015 NASA Roadmaps sections 4.5.2, 4.5.3, 11.4.1, and most significantly in 11.4.5. These objectives call on leveraging better techniques such as machine learning in the design process for the purposes of exploring the global design space, quickly re-evaluating processes due to changing conditions, reducing mission simulation time, and giving more information to mission designers. Furthermore, developing additional assets that utilize machine learning will facilitate development on current and near-future missions. Since implementing a MORL algorithm to aid in the trajectory design process directly addresses the needs specified in the 2015 Technology Roadmaps, it is then also relevant to NASA goals.

This research project addresses the objectives specified in the 2015 NASA Roadmaps sections 4.5.2, 4.5.3, 11.4.1, and most significantly in 11.4.5. These objectives call on leveraging better techniques such as machine learning in the design process for the purposes of exploring the global design space, quickly re-evaluating processes due to changing conditions, reducing mission simulation time, and giving more information to mission designers. Furthermore, developing additional assets that utilize machine learning will facilitate development on current and near-future missions. Since implementing a MORL algorithm to aid in the trajectory design process directly addresses the needs specified in the 2015 Technology Roadmaps, it is then also relevant to NASA goals.

This is early/mid-stage (TRL 3) — the most realistic path in is NASA SBIR/STTR, which funds small businesses and research institutions to develop technology aligned with NASA's needs (equity-free, phased funding). Check whether a current SBIR/STTR solicitation topic overlaps with this project's technology area, or contact the project directly (above) to ask.

None of these are guaranteed paths for this specific project — TechPort itself doesn't have an "apply" button. Reaching out to the contact(s) above with a specific question is usually the fastest way to find out what's actually open.