Presentation #405.04 in the session Machine Learning and Software Tools for Solar Physics.
The DKIST Python tools aim to lower the barrier of entry to scientific discovery with the DKIST level one data. The tools provide standard Python interfaces to the data and coordinate information in a way that enables interoperability with the wider Python ecosystem. The user tools are based on widely used Python packages such as Dask and matplotlib as well as the SunPy and Astropy packages.
The level one data being provided by the DKIST data center presents some challenges for conventional access and analysis. A single level one dataset can comprise many thousands of FITS files, to help with processing these a metadata only ASDF file is also provided. In this talk we will discuss the reasoning behind this and the functionality enabled by this ASDF file.
We shall also present the lastest version of the DKIST Python tools which include functionality for working with the ASDF and FITS files that comprise the level one data. We shall also demonstrate searching the DKIST Data Centre, retrieving metadata about datasets in ASDF format, downloading whole datasets or subsets as collections of FITS files, and loading data from large numbers of FITS files efficiently and transparently, using the Python tools.