4D Adaptive Sensing: Optimizing Sensor Deployment for Dynamic Scene Reconstruction Under Resource Constraints

Humans are constantly trying to model the changing 3D universe. In space exploration, for example, image captures from surface robots and satellites may help to better model and characterize dynamic phenomena on distant planets, such as recurring slope lineae on Mars or active volcanic sites on moons of Jupiter. However, the challenge of optimally balancing fixed camera resources to capture both spatial and temporal changes in a scene remains unsolved. In particular, there is a fundamental challenge of balancing, with fixed camera resources, exploration, painting a more comprehensive 3D scene understanding, and exploitation, repeatedly capturing the areas that change most often. Therefore, deciding how, where, and when to capture dynamic phenomena, especially under rigid constraints of space exploration technologies, remains a task requiring significant expert insight. Through this research, I propose a system for intelligent real-time camera deployment for dynamic scene modeling under constraints. Given a changing 3D scene (e.g. growing slope lineae on Mars) and a set of sensors (satellites, surface robots) with specific qualities (resolution, capture area) and constraints (latencies, locations), the key objective is to choose the best camera deployment time and location to maximize the accuracy of the reconstructed dynamic scene. I propose extending two areas of research to develop this system. First, I will extend the area of neural scene representations--using neural networks to model 3D scene geometry and color--to incorporate the novel objective of representing time-evolving 3D (4D) scenes. Second, I will extend the area of restless multi-armed bandit (RMAB) modeling--the preferred models for real-world resource allocation tasks--to incorporate the novel objective of deploying cameras to model 4D scenes. I propose a three phase plan for this work. In Phase 1, I will use neural scene representations to develop a simple dynamic scene modeling approach. Then, I will evaluate the performance of a simple RMAB-based camera deployment model, testing the reconstruction accuracy of the combined modeling-planning system in simulated Blender environments. In Phase 2, I will extend both the modeling and deployment formulations to consider real-world constraints, such as data-scarce settings, deployment location/cost limitations, and camera resolution and scale differences. I will then evaluate the performance of the modeling-planning system under these unique constraints in simulated environments. In Phase 3, I will evaluate the performance of my system in a ground deployment setting through the visiting technologist experience. Ultimately, through this project, I hope to lay the groundwork for interdisciplinary research in robust camera deployment for dynamic scene modeling, especially for high-stakes, real-time space exploration missions.

This is early/mid-stage (TRL 2) — the most realistic path in is NASA SBIR/STTR, which funds small businesses and research institutions to develop technology aligned with NASA's needs (equity-free, phased funding). Check whether a current SBIR/STTR solicitation topic overlaps with this project's technology area, or contact the project directly (above) to ask.

None of these are guaranteed paths for this specific project — TechPort itself doesn't have an "apply" button. Reaching out to the contact(s) above with a specific question is usually the fastest way to find out what's actually open.