S-PRINT: Development and Validation of a Tool to Predict, Evaluate, and Mitigate Excessive Workload Effects

The purpose of this project was to research and model human performance in unexpected workload transitions. These situations, when addressed by fatigued astronauts, constitute worst case scenarios that require specific, in-depth investigation. The project addressed two NASA risk areas -- the risk of performance errors due to fatigue, and the risk of poor task design. In addition, the research provided input for the Human Automation/Robotic Interaction research area at NASA. The project conducted integrative reviews of what was known and unknown, addressed where there was insufficient or conflicting existing research or theory for adequate quantitative prediction, and combined these insights to produce a new mechanism to understand and predict performance. This last goal was accomplished through the development of a prototype model-based tool to be used by NASA human performance researchers, automation system designers, mission planners, and astronauts to evaluate predicted astronaut performance on long-duration space missions during unexpected workload transition scenarios. The tool enables users to identify the effects of astronaut fatigue, automation system design, and task factors on predicted astronaut performance in unexpected off-nominal events (e.g., automation failures or other emergencies). The tool developed in this effort, called the Space Performance Research Integration Tool (S-PRINT), leverages the Improved Performance Research Integration Tool (IMPRINT) human performance modeling environment, and tailors it to space mission applications.

IMPRINT was developed for the Army Research Laboratory, and is available, free of charge, to U.S. government agencies. IMPRINT includes algorithms to study performance shaping factors such as fatigue, training, and use of protective clothing with human performance models. IMPRINT includes a sophisticated model of operator workload based on multiple resource theory.

S-PRINT was developed based on extensive literature reviews and meta-analyses on fatigue, automation failure response, and workload overload. We systematically evaluated human-in-the-loop research to identify and quantify factors in long-term space missions that affect astronaut workload, fatigue, and performance. The results of these meta-analyses were used to update IMPRINT algorithms so they more accurately reflect space-specific conditions. In particular, the algorithms contain empirically validated models of sleep-related fatigue, automation failure management, and multitasking in workload overload situations. Further, the team conducted a series of focused human-in-the-loop studies to address specific components of performance that were not answered in the meta-analyses. The S-PRINT tool was developed so it can be used to evaluate performance in missions that are being planned or missions that are currently underway.

The team identified two scenarios of interest for a prototype application of the tool. The primary scenario includes a single astronaut manually controlling a robotic arm when a failure occurs in the environmental process control system. When the astronaut notices the process control failure, s/he needs to prioritize among the different tasks. We worked with NASA subject matter experts (SMEs) to develop human performance models that reflect those situations. The tasks combined in this specific type of situation were of sufficient complexity to be beyond the scope of previously existing models. The team conducted an empirical, human-in-the-loop validation study of the robotic and process control system tasks. A second scenario, involving fire detection and suppression systems, was also implemented and included with the S-PRINT tool. Our scenario development and research efforts focused specifically on worst-case situations: rapid workload transitions (e.g., automation failures, other off-nominal events) resulting in overload, with a single astronaut.

The S-PRINT tool offers users access to the underlying IMPRINT modeling environment. Users who are familiar with human performance modeling can build their own, customized scenarios, and are not limited to the two scenarios developed for this project. S-PRINT also provides an easy-to-use interface in the form of data entry screens that guide the user through the process of building a scenario. It allows the researchers who are not modeling experts to specify numerous relevant factors, e.g., operators, tasks, automation support, use of protective clothing, and sleep history. The output of the model run (for both customized IMPRINT models and S-PRINT-specific models) includes parameters of interest such as operator workload, fatigue effects on task completion time and task accuracy, time to initiate tasks, time to complete tasks, results of task failures, and overall mission times, which can be used to compare relative success.

The effort to provide a new, integrative framework proved highly successful. An empirical validation study of visual attention allocation predicted by the task overload model revealed high correlations (r > 0.95) with actual human performance. This research provided a validated model-based tool to help NASA researchers evaluate potential long-duration missions, identify vulnerabilities, and test potential mitigation strategies to help ensure effective performance and safe space missions. The tool and associated scientific advances offer important insights both for future space scenarios, and for a wide range of other real-world situations.

The S-PRINT project offers several potential benefits to human performance research and industry on Earth. These include contributions to the human performance literature and the development of a model-based tool to predict operator performance in workload transitions.

The project resulted in numerous (19, as of March 2015) publications in peer-reviewed professional journals and conference proceedings. These publications describe different aspects of the human performance research conducted under the grant. In particular, two extensive meta-analyses were conducted to examine 1) the effects of sleep-related fatigue on complex task performance, and 2) the effects of task factors on task management, task switching and task shedding in overload situations. These analyses provided an empirically-derived basis for algorithms that were developed and implemented in the Improved Performance Research Integration Tool (IMPRINT) human performance modeling environment and, in particular, response to unexpected automation failures.

In addition to the meta-analyses, the project included multiple human-in-the-loop (HITL) studies to investigate 1) human-automation interaction (exploring, in particular, the effects of automation design factors and failure types on automation bias and complacency), 2) multitasking in overload situations. The results of these studies have been published and contribute to the scientific knowledge in human-automation interaction and human performance in overload. These two topic areas are relevant in numerous Earth-based domains.

The second major contribution of the S-PRINT project was the development of component models to predict 1) the effects of fatigue (i.e., due to sleep deprivation, sleep restriction, sleep inertia, and circadian cycle effects) on task completion time and task accuracy, 2) the effects of automation design factors (e.g., reliability, degree of automation, or function allocation) on operator performance, 3) the effects of failure type on operator performance, and 4) the effects of task factors (i.e., salience, expectancy, effort, and value) on task selection in overload. These models have been implemented in IMPRINT. All of these areas are relevant in Earth-based industries that require around-the-clock operations, involve the use of automation, and offer the potential for situations that put an operator in overload conditions. Examples include medicine, process control, military operations, and transportation.

This is early/mid-stage (TRL 6) — the most realistic path in is NASA SBIR/STTR, which funds small businesses and research institutions to develop technology aligned with NASA's needs (equity-free, phased funding). Check whether a current SBIR/STTR solicitation topic overlaps with this project's technology area, or contact the project directly (above) to ask.

None of these are guaranteed paths for this specific project — TechPort itself doesn't have an "apply" button. Reaching out to the contact(s) above with a specific question is usually the fastest way to find out what's actually open.