Adaptive Augmented Reality Eabled Electronic Procedure Toolset

The proposed research is aimed at investigating the feasibility to provide an integrated tool suite for development of Adaptive user interfaces for Augmented Reality (AR) enabled electronic Procedures. Human Space program has relied on procedures to operate efficiently and safely any spacecraft systems since its inception. Over the past few years, efforts to enhance these procedures using Augmented Reality has demonstrated the possibility to increase crew autonomy from ground support by providing improved guidance and just in time training. Many challenges persist in the following areas: difficult and time consuming development of the AR material, a range of potential AR hardware devices and software platforms to deploy on, and limited flexibility of the User Interface causing underwhelming user experience. Tietronix proposes to develop a toolset that will support the development of robust and adaptive user interface for the electronic procedures that leverages the power of AR technology. The envisioned toolset combines an authoring environment that enables the user to create electronic procedures with adaptive user interfaces which are dependent on their context of-use defined in terms of the user, platform, and environment. The UI can adapt to multiple elements such as the context in which the procedure is executed (workload, stressful conditions), the user skill level, the deployment platform (AR headset type, VR environment, tablet, phone), the type of cues to be provided. The use of this toolset will enable procedure developers to provide to operators enhanced situational awareness during the execution of the procedures by overlaying additional information such as instructions or graphical cues on top of the target system views, and auditory or tactile inputs. The proposed integrated suite of tools will provide critical technologies needed to develop adaptive UI for AR enabled electronic procedures, and execute these within the context of NASA space program.

The proposed AUI/AR-eProc toolset can be applied to a large set of NASA programs and tasks. NASA's vision to support human spaceflight missions beyond Low Earth Orbit will require game changing operational toolsets for training and assistance for maintenance operations. With time delays and potentially sparse communications back to earth, astronauts will face the daunting task of operating and maintaining numerous systems that might unexpectedly fail or may even be required to perform life-saving surgery without support from the earth based operators. Currently, complex assembly and maintenance tasks on-board the International Space Station are good targets for the AR-enabled electronic procedures that our proposed toolset will help create. The Autonomous Mission Operations project is interested in the use of Augmented Reality technology. The Cis-Lunar NextSTEP program is exploring the potential use of multiple advanced technologies. Our working relationship with these programs will enable us to assess the infusion of our proposed concept within these projects.

Complex Assembly and Maintenance tasks in industrial environments are excellent target domains for Augmented Reality electronic Procedures. The AUI/AR-eProc toolset has the potential to be adopted by a large number of organizations across many industries. Wherever there are complex and expensive maintenance tasks, which need to be performed safely with high accuracy, the proposed toolset can be effective at supporting the need to improve the development of advanced maintenance operating procedures. Such industries include the Aerospace maintenance services, the Oil and Gas industry as well as the automobile and heavy machinery industry. Additionally, Medical procedures are also perfect target for this technology. As Tietronix is already working with multiple large medical institutions, there is a solid potential for infusion of this technology in healthcare applications such as ultrasound guidance, dental procedure or even surgery.

This is early/mid-stage (TRL 5) — the most realistic path in is NASA SBIR/STTR, which funds small businesses and research institutions to develop technology aligned with NASA's needs (equity-free, phased funding). Check whether a current SBIR/STTR solicitation topic overlaps with this project's technology area, or contact the project directly (above) to ask.

None of these are guaranteed paths for this specific project — TechPort itself doesn't have an "apply" button. Reaching out to the contact(s) above with a specific question is usually the fastest way to find out what's actually open.