Reinforcement Learning For Coordination And Control of Swarming Satellites

Inspired by frequent observation of repetitive learned swarm behavior exhibited in nature, this novel program will develop and demonstrate new capabilities in decentralized control of large heterogeneous vehicle swarms limited in communication, sensors, and actuators, with direct application to communication-less coordination. These goals are accomplished through the adaptation and use of Reinforcement Learning solutions to the optimal control problem. Reinforcement Learning approaches define a value function, which represents the total reward for possible actions at a given state, deriving a decentralized formulation for each agent in a Multi-Agent System. The proposal implements the policy gradient method for Reinforcement Learning applied to swarming spacecraft control. Three major tasks are proposed for the development of swarming space vehicle coordination and control: Approximate Optimal Control for Large Swarms, Communication-Less Swarm Coordination Implementation, and Human-Swarm Interactions via Supervised Reinforcement Learning. Algorithm development in Phase I will extend to a Centralized Optimal Control Solution, Inverse Reinforcement Learning for the Local Decentralized Problem, Model Free Learning, "Expert Solution" Conversions to the Local Modified Local Interaction, Inverse Learning for Behavior Determination and Classification, Hyman Designed Dynamic Reward Functions, and Keep Out Zone Models. Follow-on efforts will are proposed for full implementation of the Reinforcement Learning swarm technology for real-time integrated system use and mission integration, including laboratory demonstrations of small robotic units, and the development of flight-qualified software and hardware packages for full integrated technology demonstrations.

NASA applications consist of facilitating precise, autonomous coordination of swarms of space vehicles in Earth orbit and beyond LEO, eventually into deep space. Enhanced capabilities in precise vehicle control in Multi-Agent Systems is included. The Reinforcement Learning based swarming vehicle control algorithms and integrated software for on-board implementation for future planned and upcoming multi agent missions will provide reduction of mission risk, expanded mission planning and analysis capabilities, and significant reduction in inter-agent communications requirements. The system will offer significant value and cost savings by either augmenting or replacing current relative navigation and control technologies, and has potential for reduction in support costs and system station-keeping down-time. The proposed system's algorithmic approach and software capabilities holds key mission enabling and enhancing benefits for swarms of planetary rovers, Earth orbiting swarms, and exploration missions to asteroids and comets.

Non-NASA applications for the proposed technology include significant increases in the coordination and control of large fleets of unmanned aerial systems. This has direct application to Search and Rescue operations conducted by local or municipal first response or Department of Homeland Security teams. Inter-team coordination of autonomous, robotic land, sea, and aerial vehicles are a further application, enhancing Department of Defense capabilities in reducing communication and relay limitations. Formation control via Reinforcement Learning can also benefit commercial telecommunications satellite providers maintaining growing constellations of vehicles operating as nodes in inter-satellite networks for high data rate transfers.

This is early/mid-stage (TRL 3) — the most realistic path in is NASA SBIR/STTR, which funds small businesses and research institutions to develop technology aligned with NASA's needs (equity-free, phased funding). Check whether a current SBIR/STTR solicitation topic overlaps with this project's technology area, or contact the project directly (above) to ask.

None of these are guaranteed paths for this specific project — TechPort itself doesn't have an "apply" button. Reaching out to the contact(s) above with a specific question is usually the fastest way to find out what's actually open.