Non-intrusive Hazardous Pilot Cognitive State Assessment via Semi-Supervised Deep Learning: CSA-Deep

In aviation history, many crew-related errors are caused by crew members being in hazardous cognitive states, such as overstress, disengagement, high fatigue, and ineffective crew coordination. To improve aviation safety, it is critical to monitor and predict hazardous cognitive states of crew members in a non-intrusive manner for designing mitigation strategies. In Next Generation Air Transportation System (NextGen) flight deck, emerging technologies will enable a transition from ground based navigation infrastructure to satellite based navigation and some control relating to separation of traffic will be delegated to the cockpit from Air Traffic Control (ATC). While the NextGen system will bring tremendous advantages in operational efficiency, the responsibilities of the pilot are expected to dramatically increase, which makes the hazardous cognitive state assessment even more critical. To address the above challenges, Intelligent Automation, Inc. (IAI), along with the Operator Performance Lab (OPL) in University of Iowa and Old Dominion University, proposes a real-time hazardous pilot Cognitive State Assessment system, called CSA-Deep, in all phases of flight for Integrated Crew-System Interaction (ICSI). The key innovation of the proposed research is the modeling and adaptive updating of hazardous cognitive states using a large amount of unlabeled data through semi-supervised deep learning.

We anticipate that by the end of our Phase II effort we will have matured our CSA-Deep assessment system sufficiently and we will be able to demonstrate the proposed approach to evaluate hazardous cognitive states and crew coordination in a flight simulation environment. We will work closely with NASA to ensure that technology developed in this effort can be integrated with the NASA's systems, reducing risk associated with technology transition. The proposed innovation has applicability to crew coordination research work in the Variable Autonomy Interface System (VAIS) element within NASA's Safe Flight Deck Systems and Operations (SFDSO) sub-project within the Aviation Safety Program (AvSP) Vehicle Systems Safety Technologies (VSST) Robust AutomationCrewSys (ACS) program as well as potential for enhancing safety through crew coordination in other manned aircraft and spacecraft work.

We envisage the following target markets and probable customers for the Phase 2 product: Air Operation Centers (Air Force, Navy and FAA): There is significant demand for evaluating the pilot cognitive state and coordination among crew members for air traffic management in military and civilian arenas. Furthermore, the gradual rollout of FAA's Next Generation Air Traffic System (NGATS) and its new systems, applications and operational procedures will require the development of advanced coordination assessment systems. This system could be used to perform aviation missions more effectively, as well as increasing the flight NextGen capacity. Emergency Response Systems (Homeland Security): The CSA-Deep could be used in the assessment of the operator functional state and the quality of team coordination of emergency first responders, emergency medicine personnel in procedures and exercises for disaster preparedness and mass casualty event.

This is early/mid-stage (TRL 3) — the most realistic path in is NASA SBIR/STTR, which funds small businesses and research institutions to develop technology aligned with NASA's needs (equity-free, phased funding). Check whether a current SBIR/STTR solicitation topic overlaps with this project's technology area, or contact the project directly (above) to ask.

None of these are guaranteed paths for this specific project — TechPort itself doesn't have an "apply" button. Reaching out to the contact(s) above with a specific question is usually the fastest way to find out what's actually open.