The Station Manipulator Arm Augmented Reality Trainer

One of the most demanding and high-stakes crew tasks aboard the International Space Station (ISS) is the capture of a visiting spacecraft by manual operation of the Space Station Robotic Manipulator System (SSRMS, or Canadarm2). The cost of a missed capture or improper arm/vehicle contact is likely to be very high. Since these operations may be performed up to six months after the most recent ground-based training, crews aboard the ISS prepare for such manual robotic tasks with the Robotics On-Board Trainer, a laptop-based graphical/dynamic simulator using NASA Dynamic Onboard Ubiquitous Graphic (DOUG) software from Johnson Space Center's Virtual Reality Laboratory. This system, however, does not utilize any real-world, 3-D, out-the-window views. Building upon recent advances in head-mounted augmented reality systems, the team of Systems Technology, Inc. and Dr. Stephen Robinson of UC Davis propose the Station Manipulator Arm Augmented Reality Trainer (SMAART) that will offer ISS crews significantly more realistic on-board refresher training for vehicle capture by manipulating the actual SSRMS with real out-the-Cupola-window views, but with a graphically-simulated vehicle overlaid on the astronaut's non-simulated view via a head-mounted display. Providing multi-sensory realism in on-board training for such high cognitive-demand skills is expected to increase crew readiness and therefore reduce operational risk for visiting vehicle capture.

The proposed Station Manipulator Arm Augmented Reality Trainer (SMAART) technology directly supports the Augmented Reality topic of the NASA Space Technology Mission Directorate (STMD) by allowing astronauts to train/mission rehearse the complex resupply capsule capture task with the mission specific hardware. Currently all training conducted once on-board the ISS is accomplished using a simulation program that does not include use of the actual SSRMS inceptors or feature real-world views. This proposed program also compliments the current STMD Game Changing Development Program IDEAS - Integrated Display and Environmental Awareness System. This program led by Kennedy Space Center with support from Ames Research Center is developing a transparent head mounted display augmented reality system to support situational awareness for operations on Earth and in Space. The emerging SMAART technology can be used to support and/or enhance IDEAS-based applications.

The SMAART-based technology is a leap forward from conventional augmented reality in that it will allow the virtual objects to interact directly, not passively with real-world objects in the combined mixed-world environment. Furthermore, using the new geometry-based keying technology, any vehicle or environment can potentially become an in situ simulator. Examples include aviation, a shipyard crane cab, a UAS ground station, and many others. Imagine that a naval aviator can repeatedly practice carrier landings in the actual aircraft at a safe altitude to improve proficiency in a wide variety of conditions before ever attempting actual landings on the carrier. This technology will find applications in the area of hazardous duty training for first responders and the military and broad commercial applications in training for industrial maintenance tasks and operator training with complex machines and vehicles.

This is early/mid-stage (TRL 3) — the most realistic path in is NASA SBIR/STTR, which funds small businesses and research institutions to develop technology aligned with NASA's needs (equity-free, phased funding). Check whether a current SBIR/STTR solicitation topic overlaps with this project's technology area, or contact the project directly (above) to ask.

None of these are guaranteed paths for this specific project — TechPort itself doesn't have an "apply" button. Reaching out to the contact(s) above with a specific question is usually the fastest way to find out what's actually open.