NexTrack: sun tracking through cloud edges for next-generation sunphotometry using a multi-spectral machine vision system

The radiative effect of atmospheric aerosol particles, and their interactions with clouds, represent the largest uncertainty in the Earth's radiative energy budget. The global atmospheric aerosol burden is measured from NASA’s orbital assets such as the MODIS sensors flying on the Terra and Aqua spacecraft, validated by systematic ground-based sunphotometer measurements. The NASA Ames 4STAR airborne sunphotometer allows measurement of aerosol properties by suntracking and sky-scanning, as well as cloud properties while flying underneath a cloud. In order to remain the world leader in airborne sunphotometry, Ames SunSat group is developing the next-generation airborne instrument, as well as exploring derivative instruments for alternative mobile platforms such as unmanned airborne systems (UAS) and ships. To enable new science opportunities for observation of aerosol-cloud interactions at the boundary between clear sky and cloud, we would like to improve the tracking subsystem to allow our new instruments to track the sun as far as possible past the clear-sky/cloud boundary. Measurements such as these have previously been explored, however there remain some questions around sunphotometer performance in these measurement modes. One of the problems is retaining accurate sun tracking during the transition between clear sky and cloud, as the cloud optical depth increases and the sun becomes partially, and then completely obscured by the cloud; and forward scattering by cloud droplets leads to the "bright object" in the tracking system's field of view no longer being circular. This work proposes a threshold requirement to implement single-camera sun-tracking using a machine learning approach to enable tracking while the sun is obscured by a randomly shaped object such as a cloud, which potentially fragments the circular solar disk, and presents bright areas of forward scattering which confuse a conventional sun-tracking system. In order to reach our proposed baseline requirement of low-contrast tracking behind thin clouds, we take advantage of the increased oxygen-A (O2-A) band absorption around 765nm, of photons that have been multiply-scattered inside a cloud, thereby increasing the absorption path-length. This work propose that spatially resolving the degree of O2-A band absorption around the solar disk using a dual-camera dual-wavelength system, will enable tracking of the sun in low-contrast scenes behind a thin cloud. The effort will build an optical system with on-band and off-band optical band-pass filters placed in front of two commercial off-the-shelf (COTS) near-infrared cameras with synchronized frame rates. A shutter on each camera will enable measurement of dark current from each camera. This will be mounted to one of our sun-tracking ground prototype robots. Low-cost low-power COTS graphics processing unit (GPU) hardware such as Nvidia's Jetson is now available for machine learning applications, in a form-factor that is realistic for building into future sunphotometers.

To enable new opportunities for sunphotometry in aerosol-cloud interaction studies we would like to be able to track the sun from a moving platform, past the clearsky/cloud boundary. Camera-based sun tracking systems for solar energy applications, typically use a monochrome image to distinguish the sun using image processing techniques such as a Hough transform. We will build a multi-spectral machine vision system which uses a convolutional neural network to identify the centroid of the sun, while obscured behind a cloud.

This is a mature technology (TRL 7+) — the realistic path in is usually NASA's Technology Transfer Program: licensing an existing NASA patent, or a Space Act Agreement to use NASA facilities/expertise directly. NASA also runs a startup licensing program with no upfront fee for companies formed to commercialize a specific NASA technology.

None of these are guaranteed paths for this specific project — TechPort itself doesn't have an "apply" button. Reaching out to the contact(s) above with a specific question is usually the fastest way to find out what's actually open.