Instance-Aware Semantic Mapping for Robotic Caretaking in Space Habitats

Autonomous robotic caretaking is an emerging technology crucial for future deep space habitats such as Gateway, the Artemis BaseCamp, and future Mars missions. Mobile inspection is one key robotic caretaking function and a major use for the Astrobee free-flyerrobots on the International Space Station. Intra-vehicular robots must be endowed with perception capabilities comparable to thoseof people to achieve autonomous inspection and caretaking of human habitats. We propose object detection, instance-aware semanticsegmentation, and mapping capabilities for autonomous fault detection.We first develop a framework to infer deformable object class, segmentation, and shape completion under occlusion using deeplearning to enable perception of anomalies among wiring comprising space habitat power and communication networks. Next, wegeneralize object class inference and segmentation in a scene image by introducing instance-aware semantic mapping of all objectsin the scene to the perception framework. The ability to perceive and map individual objects in a scene is important for anomalydetection and intervention: if a robot understands which specific component of a large engineering system experienced a failure, itcan reference prior knowledge about how the failure impacts related objects and promptly enter a fault recovery state of operation.Finally, a library of scene information including the number and types of objects in the scene and how objects are related to each otherwill be constructed. The information contained in this library for a given scene will be compared with the scene information abstractedfrom the instance-aware semantic map to demonstrate end-to-end autonomous fault detection. A robot endowed with this capabilitywill be able to efficiently react to anomalous environmental configurations to ensure the safety of the habitat.

This is early/mid-stage (TRL 2) — the most realistic path in is NASA SBIR/STTR, which funds small businesses and research institutions to develop technology aligned with NASA's needs (equity-free, phased funding). Check whether a current SBIR/STTR solicitation topic overlaps with this project's technology area, or contact the project directly (above) to ask.

None of these are guaranteed paths for this specific project — TechPort itself doesn't have an "apply" button. Reaching out to the contact(s) above with a specific question is usually the fastest way to find out what's actually open.