Robot Learning and Autonomy with Reduced Supervision through Active Reward Design

Building robots that act safely around astronauts and as intended when deployed with limited operator supervision depends on designing good reward functions for these robots to optimize. Unfortunately, designing reward functions is difficult, and hence rewards are often misspecified. This can elicit undesirable behavior, and even cause harm to the robot, or astronauts around it. We propose to develop robots that actively query humans for information about the desired reward function when needed, so as to complete their task safely and as intended with minimal supervision. To that end, we will develop an active reward design framework enabling the robot to query the human for information about the desired reward through various modalities, such as demonstrations, corrections, preferences, or full reward functions. We plan to experimentally validate this framework on both simulated and real robotic platforms in two settings. The first is a robot that operates in close proximity to human astronauts, such as Astrobee, that is set to be deployed on the International Space Station. The second is a robot, such as the K-REX planetary rover, operating on a distant planet so that operator supervision is necessarily limited. In this case, the robot will learn through a limited number of queries, constrained, for example, by the bandwidth available.

Building robots that act safely around astronauts and as intended when deployed with limited operator supervision depends on designing good reward functions for these robots to optimize. We propose to develop robots that actively query humans for information about the desired reward function when needed, so as to complete their task safely and as intended with minimal supervision.

This is early/mid-stage (TRL 3) — the most realistic path in is NASA SBIR/STTR, which funds small businesses and research institutions to develop technology aligned with NASA's needs (equity-free, phased funding). Check whether a current SBIR/STTR solicitation topic overlaps with this project's technology area, or contact the project directly (above) to ask.

None of these are guaranteed paths for this specific project — TechPort itself doesn't have an "apply" button. Reaching out to the contact(s) above with a specific question is usually the fastest way to find out what's actually open.