Using Transformers to Improve Ionospheric Modeling and Enable a Wider Input Space

Understanding the ionosphere is crucial for many aspects of modern life, especially regarding communications. However, the impact of the ionosphere depends heavily on the electron density, which is difficult to measure densely, leading to a need for models. While using correlated solar activity metrics can help, using direct measurements is challenging due to inconsistently structured data, such as ungridded or arbitrary-length formats, which hinder the use of many machine learning techniques. This applies to electron density measurements, which are sparse and scattered, as well as Total Electron Content (TEC) (the integral of electron density) measurements, which are inconsistent in amount and location at any one time.

Such drawbacks could be addressed by transitioning data into standardized sizes, for which we have developed a technique. By using transformer-like architectures, we can embed arbitrary lengths of sequences into fixed-length vectors that can be further used in other models. Our work has demonstrated that we are able to capture details of inconsistent data, especially TEC, which enables us to use recent TEC history to drive improved 3D electron density predictions. We also compare our approach with more traditional masked CNNs and highlight the benefits of using an attention-based approach like ours. Since our approach is not confined to a grid, we can even use recent electron density history measurements from moving spacecraft, an input previously unexplored.