Adaptive Sampling to Enable Autonomous Systems Analysis and Design

The proposed approach is to develop the enabling software tools necessary to shift our data collection process to an adaptive method that responds to new information gained during the gathering process. For example, when building a metamodel from the data collected during an experiment or design activity, new samples should be targeted in regions of the information space where the model's uncertainty or instability is the greatest, not in regions that are relatively well characterized or invariant. This will allow project resources to be more precisely targeted to achieve a better value with respect to knowledge gained, rather than relying on a scatter-shot approach that DoE often resembles. While the core idea of collecting data adaptively is not new (and traces back to at least the 1980's via Sacks, et al.for Kriging methods of metamodeling), the proposer has observed that using an adaptive approach is rarely seen during analysis in engineering practice, even if the benefits may be significant. Hypothetically, this may be due to the sometimes esoteric (but sound) theory involved and the fact that many end-usertools may be incompatible with the sometimes-chaotic, fast-paced process of engineering system design. However, properly designed, generalized software tools with engineering in mind may push adaptive sampling over a final hurdle to wide-spread adoption and may also enable autonomous data collection methods where acomputer agent can assist or be responsible for exploring any given information space. \n\nIt is the proposer's observation that there is both industry and government interest in software to aid in adaptive sampling, evidenced by personal conversations at, and correspondence after conferences. Interest has also been expressed by collaborators on programs of record during technical interchange meetings. Interested governmental parties outside of NASA include DoD and Air Force civilian and active duty personnel. Therefore, development in this area may position NASA to be an important player at a national level in deploying these methods.\n\nBringing adaptive sampling tools into common practice will:\n\n\xb7 Foster team communication by having more frequent conversations about the data being collected and avoiding the throw it over the fence mentality.\n\n\xb7 Improve metamodel confidence by seeing if metamodels are converging, similar to how we approach computational fluid dynamics simulations.\n\n\xb7 Catch issues earlier by detecting unusual system behavior, if nonphysical, and correcting for future data samples.\n\n\xb7 Better allocate project resources so data collection can stop when it no longer suits project objectives, or to make high intensity tasks such as uncertainty quantification more efficient and palatable to practitioners. \n\nSome adaptive sampling methods may not be robust to general problems or may fail by suggesting samples in regions with low informational value. A mitigation to this risk would be to validate the methods on challenging test problems and also to evaluate their behavior in parallel with more established methods on the critical path of projects. \n\nA result of the Phase I effort will be software development of a minimum viable product with core routines to drive the adaptive sampling decision making and analysis process. These core methods, if programmed in a cross-platform language such as python, could be used by other standard, proprietary software platforms such as MATLAB and JMP in order to provide a flexible environment for the end user. The effectiveness of the methods can then be compared quantitatively with state-of-the-art DoE methods on a set of benchmark cases to evaluate each in terms of efficiency and rate of convergence of different metamodel forms used.\n\nBibliography:\n1. E.L. Axdahl,Shifting Data Collection from a Fixed to an Adaptive Sampling Paradigm,MODSIM World, Norfolk, VA, April 2018\n2. E.L. Axdahl and R.A. Baurle, Application of Adaptive Sampling to Advance the Metamodeling and Uncertainty Quantification Process, DATAWorks, Springfield, VA, March 2018\n3. E.L. Axdahl, J.J. Yagle, T.D. Smith, Classification Modeling and Adaptive Sampling ofHypersonic Inlet Unstart using Logistic Regression, JANNAFCS/APS/EPSS/PSHS/PIB Joint Subcommittee Meeting, Newport News, VA, December2018

The application of physics-based modeling and simulation of complex systems is a way to improve the fidelity and credibility of engineering design. For aerospace systems, model-based systems-engineering frameworks ease the process of managing ensembles of discipline-specific design and analysis codes; however, a key bottleneck remains that iteratively running each code can take a significant amount of time, especially combined with the desire to bring high fidelity tools earlier in the design process, even to the conceptual level. This desire is driven by 80%+ of lifecycle costs for a program being fixed by design decisions made early in the development process. Therefore, value-based engineering will best be accomplished via decision making using evidence from high fidelity analysis. The goal of this effort will be to develop software tools that designers and analysts can use to collect information more efficiently than what is standard practice today.\n\nState-of-the-art data collection uses Design of Experiments (DoE), where analysts and designers rigorously define a set of variable setting combination to efficiently explore an information (or design, or uncertainty) space to better understand how system inputs map to outputs. Transitioning from brute force (i.e., fullfactorial or lookup table) data collection to DoE has paid off by allowing engineers to make quicker, better-informed decisions or database delivery. However, DoE has a drawback in not being inherently responsive to the data collection process; it is a method for pre-determining sample batches to collect before the analysis or experiment is begun with minimal regard towhat the data tells us about the shape of the information space as it is collected.

This is early/mid-stage (TRL 1) — the most realistic path in is NASA SBIR/STTR, which funds small businesses and research institutions to develop technology aligned with NASA's needs (equity-free, phased funding). Check whether a current SBIR/STTR solicitation topic overlaps with this project's technology area, or contact the project directly (above) to ask.

None of these are guaranteed paths for this specific project — TechPort itself doesn't have an "apply" button. Reaching out to the contact(s) above with a specific question is usually the fastest way to find out what's actually open.