Knowledge Transfer for Robust GeoAI Across Space, Sensors and Time via Active Deep Learning

Recent advances in optical sensing technology (e.g., miniaturization and low-cost architectures for spectral imaging in the visible, near and short-wave infrared regimes) and sensing platforms from which such imagers can be deployed (e.g. handheld devices, unmanned aerial vehicles) have the potential to enable ubiquitous passive and active optical data on demand to support sensing of our environment for earth science. One can think of the current sensing environment as a vast sensor-web of multi-scale diverse spatio-temporal data that can inform various aspects of earth science. Although this increase in the quality and quantity of diverse multi-modal data can potentially facilitate improved understanding of fundamental scientific questions, there is a critical need for an analysis framework that harmonizes information across varying spatial-scales, time-points and sensors. Although there have been numerous advances in Machine Learning models that have evolved to exploit the rich information provided by multi-channel optical imagery and other high dimensional geospatial data, key challenges remain for effective utilization in an operational environment. Specifically, there is a pressing need to have an algorithm base capable of harmonizing sensor-web data under practical imaging scenarios for robust remotely sensed image analysis. In this project, we propose to address these challenges by developing a machine learning algorithmic framework and an associated open source toolkit for robust analysis of multi-sensor remotely sensed data that advances emerging and promising ideas in deep learning, multi-modal knowledge transfer between sensors, space and time, and provides capability for semi-supervised and active learning. Our model would seek to harmonize data from heterogenous sources, enabling seamless learning in a disparate ensemble of multi-sensor, multi-temporal data. Our proposed architecture will be comprised of a generative adversarial learning-based knowledge transfer framework that will use optics inspired and sensor-node specific neural networks, multi-branch feed forward networks to transfer model knowledge from one or more source sensor nodes to one or more target sensor nodes, and semi-supervised knowledge transfer. This game-changing model transfer and cross-sensor super-resolution/sharpening capability will enable end-users to leverage training libraries that provide disparate or complementary information (for example, imparting robustness to spatio-temporal non-stationarities and enabling learning from training libraries from different geographical regions, sensors, times and sun-sensor-object geometries). We will develop and validate active deep learning capability within our knowledge transfer framework that will seek to strategically facilitate additional labeling in the source and/or target sensor-nodes for further improving performance. A functional prototype of our framework will be implemented on a commercial cloud for dissemination and access to stake-holders and the broader research community. Our algorithms will be developed with a specific earth science application focus – earth observation based agricultural sensing. However, the tools developed in this project will be readily applicable to other domains, and will have far reaching benefits to NASA earth science – including ecological impacts of climate change, forestry, wetlands, etc., using a wide array of spaceborne data sources such as: (1) multispectral imaging systems (e.g. Landsat, Sentinel), (2) imaging spectroscopy (e.g. DESIS and the future HyspIRI and CHIME missions), (3) SAR (Sentinel, (future) NISAR), and a rich archive of NASA, ESA and commercial satellite imagery, as well as airborne platforms (e.g. AVIRIS-NG, G-LiHT, commercial). The proposed capability will also be important in successful analysis of data acquired by constellations of satellites, for which seamless learning is a key objective.

Advance Earth system science knowledge through the Identification, develop, and demonstrate innovative information systems technologies

This is early/mid-stage (TRL 3) — the most realistic path in is NASA SBIR/STTR, which funds small businesses and research institutions to develop technology aligned with NASA's needs (equity-free, phased funding). Check whether a current SBIR/STTR solicitation topic overlaps with this project's technology area, or contact the project directly (above) to ask.

None of these are guaranteed paths for this specific project — TechPort itself doesn't have an "apply" button. Reaching out to the contact(s) above with a specific question is usually the fastest way to find out what's actually open.