Robust Voice Communication Understanding for Single-Pilot Operations

Motivated by the anticipation of pilot shortage in the future and the quest of cost reduction in airline operations, the single-pilot operation (SPO) concept emerges as a promising alternative of the current-day multi-pilot operations. This proposal addresses the feasibility of constructing a spoken language understanding system for decoding voice communications in Air Traffic Control. In particular, we address the issue of developing a voice communication understanding system (VCUS) that would serve as one key component in both cockpit automation and ground-based automation for supporting the SPO concept. Leveraged from our prior development on noise-robust speech recognition system for the Navy and virtual agents for NASA to support human-in-the-loop simulations, an infrastructure of VCUS in Air Traffic Control of commercial flights will be developed. A feasibility demonstration of the VCUS that extracts out semantic information for persistent display of clearance message within the flight deck will be provided by the end of the Phase I research. Phase II work will utilize the infrastructure built in Phase I to expand the VCUS to a full-scale prototype that supports cockpit automation and ground-based automation for SPO.

The Voice Communication Understanding System (VCUS) designed for Air traffic Control is conceived as part of the cockpit automation and ground-based automation prototypes that support SPO feasibility assessment studies within NASA. In addition to supporting SPO operations, the VCUS that enables real-time and post-flight analysis of pilot-controller voice communications can directly benefit human-in-the-loop (HITL) experiments conducted at numerous NASA simulation facilities identified in an FAA/NASA NextGen Human Factors Research Coordinate Plan.

The Voice Communication Understanding System (VCUS) will benefit air traffic control training facilities, including the FAA Academy at the Mike Monroney Aeronautical Center in Oklahoma City, OK, and DoD training facilities such as the Naval Air Technical Training Center (NATTC) in Pensacola, FL. VCUS would also benefit the Army PEO Aviation, which calls for voice communication understanding capability for supporting the manned-unmanned teaming training. In addition, VCUS can be integrated with the Speech-Enabled Simulation Interface Agent (SESIA) currently under development. The integrated product can be adapted for other simulation facilities within the FAA for supporting HITL experiments.

This is early/mid-stage (TRL 4) — the most realistic path in is NASA SBIR/STTR, which funds small businesses and research institutions to develop technology aligned with NASA's needs (equity-free, phased funding). Check whether a current SBIR/STTR solicitation topic overlaps with this project's technology area, or contact the project directly (above) to ask.

None of these are guaranteed paths for this specific project — TechPort itself doesn't have an "apply" button. Reaching out to the contact(s) above with a specific question is usually the fastest way to find out what's actually open.