Combining Uncertainty Quantification and XAI to Understand the Sensitivities of Deep Learning Winter Precipitation Type Predictions

The ability to adapt to rapidly changing winter weather conditions has been a decisive factor in conflicts for centuries and continues to play a major role in global conflicts today, including the war in Ukraine and the evolving geopolitical situation in the melting Arctic. One of the major uncertainties in winter weather forecasting is when winter precipitation transitions from rain to freezing rain, sleet, and/or snow and when that transition occurs. Current NWP and heuristic based methods do not account for the uncertainty in the precipitation type predictions and struggle with freezing rain and sleet. As part of the AI2ES NSF AI Institute, the MILES group at NCAR has developed a machine learning precipitation type model that utilizes the uncertainty quantification technique of evidential neural networks. Evidential models can estimate both aleatoric and epistemic uncertainty with a single parametric model, saving considerable computational time compared with ensemble-based approaches. The ML model is trained on multiple years of NWP analysis soundings from the NOAA Rapid Refresh model and quality-controlled mPING crowd-sourced observations of precipitation type. We have found that the evidential model improves in accuracy over the RAP precipitation type algorithm for all p-types. Analysis of composite soundings has revealed that uncertainty in the prediction increases as the temperature profiles shift toward 0 C near the surface, and that the model can discriminate the p-type based on the characteristics of the near-surface warm-nose. XAI analysis of p-type predictions has revealed connections between prediction sensitivity and frontal passages and variations in terrain height, even when only using vertical profile data as input. The trained models have also been applied to the HRRR and GFS models and have shown promising results with both models at both analysis and forecast times. Currently we are working with Vaisala to test the impact of the ML precipitation type predictions in their road-weather model to understand how much value the increased accuracy and uncertainty estimates add to their downstream predictions of road conditions.