Extending MAG4 Code to New Operational Users

Large solar flares emit energetic particles, X-rays, radio noise, and coronal mass ejections (CMEs) that have wide-ranging impacts to Earth and its infrastructure. There are a number of models and observations that have been developed by government, university, and private sector researchers to try to understand and forecast solar flares. However, successfully bridging the gap between these research missions/activities and operational forecasters continues to be a challenge. In an FY18 TIP proposal, the Short-term Prediction Research and Transition (SPoRT) Center in the Earth Science Branch (ST11) partnered with scientists in the Heliophysics and Planetary Science Branch (ST13) to perform the first iteration of a transition of the MSFC- developed MAG4 product to operational forecasters. Lessons learned in the migration of this software to an operational code infrastructure and feedback from users identified advancements required before the product would be ready for operational implementation. The primary objective of this proposal is to advance the data management, graphics, and select scientific sections of the MAG4 code corresponding with specific feedback from multiple operational users in order to create a more stable deployment for operational forecasters that will advance the code to TRL6. It is expected that this work (coupled with the results from the FY18 TIP) will position MSFC to receive funding as part of NASA’s response to Congressional legislation to increase the effectiveness of transition of research into operations for space weather users.

Large solar flares emit powerful electromagnetic particles with wide-ranging impacts. Damage can occur to launching spacecrafts and on-orbit assets (e.g., GPS, telecommunication, and science observing satellites). Major solar flares (e.g. CMEs) even have the capability to disrupt power grids, resulting in widespread power outages that could last months (National Science and Technology Council, 2015). As a result, improving forecasts of solar active region (AR) threat has emerged as a critical national imperative. MSFC has developed the MAG4 (MAGnetic forecasting) model, which measures a free-energy proxy and uses recent flare activity to produce an initial forecast. MAG4 generates a magnetogram, which is a pictorial representation of the spatial variations in strength of the solar magnetic field and is used by space weather forecasters to identify ARs. MAG4 was developed using heavy investment from NASA and was named the 2016 MSFC Software of the Year and an Agency-wide runner up. MAG4 provides a quantitative probability of AR potential to produce a dangerous solar flare, which was communicated as an advantage over other approaches used by forecasters at the NOAA Space Weather Prediction Center (SWPC) during a shadowing and science sharing session in July 2018. In fact, MAG4 has been shown to more accurately forecast AR threat level within 30 degree of disk center, outperforming the subjective McIntosh AR classification that NOAA has used for decades.

An FY18 TIP proposal successfully integrated MAG4 into the real-time data processing and dissemination framework at the Short-term Prediction Research and Transition (SPoRT) project. SPoRT has more than 15 years of experience transitioning experimental Earth science research products into national terrestrial weather forecasting operations through directed funding from NASA HQ Earth Science Division and investment by the National Oceanic and Atmospheric Administration (NOAA) satellite proving ground programs. Forecaster enthusiasm in providing feedback to the proposal team during the FY18 TIP activities demonstrated that both interest in MAG4 and SPoRT’s research-to-operations/operations-to-research (R2O/O2R) paradigm of working collaboratively with users to develop solutions to forecasting challenges can be applied to space weather forecasting. In addition to the access, assessment, and science sharing outlined above, the development of user-focused training, and decision support website — designed collaboratively with SWPC forecasters—were outcomes of this first TIP. MSFC/SWPC collaborations are key for MSFC inclusion in the national strategy for space weather applications.

However, the FY18 TIP activities revealed a number of software limitations (specifically during the current solar minimum) when directly integrating the MAG4 executable into the SPoRT processing stream. These limitations produce less-than-optimal operational implementation and need to be improved. For example, if data from downstream hosts are not available, the executing code will pause for multiple hours and continue querying the input data’s servers. While this is acceptable for a researcher who generally is only interested in an active Sun, this resulted in failures of a system that is expecting constant solar input. While stopgap measures have been implemented to mitigate the impacts of missing data on both the runtime performance of MAG4 and what is communicated to users via the website, more work on the front-end logic is needed to stabilize the code for real-time execution. In addition, MAG4 code is being run by other NASA space weather stakeholders (e.g., Space Radiation Analysis Group (SRAG) at JSC and Community Coordinated Modeling Center (CCMC) at GSFC), but these partners have not updated the version of the code that they are using for many years due to the difficulties using the Interactive Data Language (IDL) executables that have been previously delivered. For example, recent developments such as the production of vector magnetograms, which allow users to more effectively analyze the magnetic fields on the solar surface, have not been incorporated into the real-time code distribution. Feedback from SWPC forecasters and users at the SRAG indicate a desire to see these latest updates incorporated in the output. Additionally, some of the MAG4 functionality for forecasting ARs is reduced because it uses a priori data from SWPC to define the solar regions it analyzes, so modifying the internal logic of the code is necessary to enhance the model as a true forecasting tool. Both SRAG and CCMC represent additional stakeholders who have a large roster of users that access models for space weather applications and can extend MSFC research expertise to the external community. SRAG and CCMC have communicated a need for the improvements noted above (see Appendix A for emails from SRAG and CCMC).

As a result, the primary objective of this proposal is to add new research results into the operational code, advance the data download/management and the graphical output for MAG4 towards a more flexible, robust, pseudo-operational software system in the Python scripting language, and transition the code to SWPC, CCMC, and SRAG. Specifically, we will incorporate code to ingest, process, and output vector magnetograms. We will modify the front-end logic related with upstream data availability, management, and use and improve the logic for how and when MAG4 runs if data are not available to enable greater real-time stability. Finally, we will modify the back end of the code to enable improved graphical output using Python plotting routines, which produce more user-friendly products compared to the current IDL outputs. SPoRT will then engage CCMC and SRAG to transition the updated code to these users. Based on an analysis by the technical Co-Is, a full transition of the entire MAG4 code (a total of approximately 60,000 lines of code) is not feasible given the scope of this proposal. However, the advancements described above will enable easier maintenance by all of NASA’s space weather R2O groups and ensure continued use by the space weather community.

Through the proposal process, we expect to increase the TRL for MAG4 from TRL5 to TRL6. The previous TIP proposal as well as previous MSFC and NASA investments have matured MAG4 to a level where it has been run and demonstrated in a relevant environment but needs the aforementioned updates to more optimally address real-time processing needs by the user groups for consideration as a fully operational product.

This is early/mid-stage (TRL 5) — the most realistic path in is NASA SBIR/STTR, which funds small businesses and research institutions to develop technology aligned with NASA's needs (equity-free, phased funding). Check whether a current SBIR/STTR solicitation topic overlaps with this project's technology area, or contact the project directly (above) to ask.

None of these are guaranteed paths for this specific project — TechPort itself doesn't have an "apply" button. Reaching out to the contact(s) above with a specific question is usually the fastest way to find out what's actually open.