Scalable ParaView for Extreme Scale Visualization

Petscale computing is leading to significant breakthroughs in a number of fields and is revolutionizing the way science is conducted. Data is not knowledge, however, and the challenge has been how to analyze and gain insight from the massive quantities of data that are generated. In order to address the peta-scale visualization challenges, we propose to develop a scientific visualization software that would enable real-time visualization capability of extremely large data sets. We plan to accomplish this by extending the ParaView visualization architecture to extreme scales. ParaView is an open source software installed on all HPC sites including NASA's Pleiades and has a large user base in diverse areas of science and engineering. Our proposed solution will significantly enhance the scientific return from NASA HPC investments by providing the next generation of open source data analysis and visualization tools for very large datasets. To test our solution on real world data with complex pipeline, we have partnered with SciberQuest, who have recently performed the largest kinetic simulations of magnetosphere using 25 K cores on Pleiades and 100 K cores on Kraken. Given that IO is the main bottleneck for scientific visualization at large scales, we propose to work closely with Pleiades's systems team and provide efficient prepackaged general purpose I/O component for ParaView for structured and unstructured data across a spectrum of scales and access patterns with focus on Lustre file system used by Pleiades.

We expect our solution to find rapid adoption among the supercomputing user base since it circumvents the well known problems associated with visualization of large data sets with minimal interruption of existing visualization environments. We expect other non-NASA groups such as NSF's Kraken and Ranger, and DOE's Jaguar, to embrace this technology.

Our tools will significantly impact large supercomputing systems such as NASA's Pleiades. Our approach build on existing NASA technology and installed visualization systems, including ParaView, and leverages work that our subcontractor SciberQuest has performed: kinetic simulations of the magnetosphere using 25K computational cores on Pleiades and 100K cores on Kraken. The major benefit of the work is that it will dramatically improve the IO bottleneck found in many supercomputing systems, including the Lustre file system used by Pleiades.

This is early/mid-stage (TRL 4) — the most realistic path in is NASA SBIR/STTR, which funds small businesses and research institutions to develop technology aligned with NASA's needs (equity-free, phased funding). Check whether a current SBIR/STTR solicitation topic overlaps with this project's technology area, or contact the project directly (above) to ask.

None of these are guaranteed paths for this specific project — TechPort itself doesn't have an "apply" button. Reaching out to the contact(s) above with a specific question is usually the fastest way to find out what's actually open.