NASA EPSCoR Rapid Response Research: Multispectral and Hyperspectral Data Representation using Deep Autoencoders and Transfer Learning of BERT for Improved Detection of Harmful Algal Blooms

The goal of this proposal is to apply artificial intelligence paradigms for developing a knowledge base of drivers of Harmful Algal Blooms (HABs) from multispectral and hyperspectral images, and train a Bidirectional Encoder Representation of Transformers by transfer learning to improve detection of HAB in Lake Erie. There has been drinking water risk due to the presence of Cyanobacteria Microcystis in Lake Erie, which is also affecting tourism, recreation, and fishing. The algal blooms show inherent optical properties that have a high variability in spatial and temporal scales. Hence, satellite, and airborne remote sensing offers a viable solution to monitoring cyanobacteria HAB. In this project, we will develop and apply unsupervised data representation using autoencoders to accurately estimate the diverse bloom composition with variable absorption and backscatter properties. We will also process multispectral Satellite Image Time Series (SITS) using the trained BERT model to predict the class distributions of HAB constituents. The objectives for this NASA Rapid Response Project are:

1) Multispectral and hyperspectral image data representation of drivers of HABs by unsupervised learning using deep Autoencoders (AE) and creation of a knowledge-base of HAB drivers. 2) Train a deep Bidirectional Encoder Representation of Transformers (BERT) by transfer learning of the knowledge-base of drivers of HABs to improve their detection.

Multispectral images from NASA’s Earth observing satellites such as Landsat 8, MODIS, and Sentinel 3 will be used in this project. Airborne Hyperspectral Imager (HSI 3.2) images with a meter spatial resolution and spectral bands ranging from 400 to 900nm will be provided by Glenn Research Center (GRC). Deep AE will be used for unsupervised feature extraction from these images to create a knowledge-base of novel drivers of HABs. The hidden layers of the AE will identify intrinsic structures in the image datasets, as well as produce a compressed representation of the multi-sensor image datasets. The BERT model consists of layers for pixel embedding, and self-attention feedforward sequence, and will be trained with the knowledge-base by transfer learning. The learnt BERT model will be used for improving predictions of HAB class distributions, and estimating fractional abundances of HABs in the hyperspectral images. The BERT trained model will be useful for supporting operational forecasting systems, and water quality management in Lake Erie. Eng. Roger Tokars, optics Engineer, GRC, NASA will collaborate in this project, giving important information on the pre-processing, and corrections done on the HSI 3.2 hyperspectral images. Dr. Jeffrey Luvall, GRC, NASA will also be collaborating in this project, and will provide guidance on the use of Ecostress and Desis images. The NASA collaborators will provide other available datasets and field measurements for validation of the results for quantifying drivers of HABs in Lake Erie and other aquatic regions. This project will be conducted at the Laboratory for Applied Remote Sensing, Imaging, and Photonics (LARSIP), ECE, UPRM. This project is conducted in a Hispanic minority serving institution with 99% Hispanic students who will gain knowledge in the fields of satellite remote sensing, multispectral and hyperspectral image processing and Python tool development. This project will directly support two graduate students in Earth science research, and involve many undergraduate students through course work and workshops. The PI will use the material from this project in the Remote Sensing, and Image Processing courses taught at ECE, UPRM. The students will be exposed to NASA Earth science related research. This project will also enable Hispanic minority students to take up internships with NASA, and add to the workforce development for future NASA missions.

This is a mature technology (TRL 7+) — the realistic path in is usually NASA's Technology Transfer Program: licensing an existing NASA patent, or a Space Act Agreement to use NASA facilities/expertise directly. NASA also runs a startup licensing program with no upfront fee for companies formed to commercialize a specific NASA technology.

None of these are guaranteed paths for this specific project — TechPort itself doesn't have an "apply" button. Reaching out to the contact(s) above with a specific question is usually the fastest way to find out what's actually open.