Presentation #119.04 in the session Promoting Open Science in Heliophysics and Space Science through Software — Poster Session.
The Daniel K Inouye Solar Telescope (DKIST) Data Center (DC) provides a central link between the data take at at the telescope and achieving DKIST’s scientific goals.
DKIST is expected to take large volumes of raw data per year (~3PB) resulting in calibrated data products of similar volume. Both the raw and calibrated data products will need to be curated for the lifetime of the observatory (two full Hale cycles). DKIST will home to five facility class instruments on the Coudé (instrument) platform. The complexity and operational flexibility of these instruments adds to the data-handling challenge. The DC will provide calibrated data to the community, which because of the data volume, requires automated calibration pipelines. Implementing the pipelines that will remove telescope, instrumental and some atmospheric seeing effects, constitutes significant challenges, but they are necessary to achieve the full scientific possibilities of DKIST, and open the data to a far wider community. Considerable effort has been directed towards ensuring the quality of the calibrated data, including focusing on recording processing provenance, including the software used, such that any data version may be reliably reproduced going forward. The implementation of the calibration pipeline is described in the poster by Watson et al., at this meeting.
In keeping with the notion of increasing the access to and utility of DKIST data, the DC has created a set of Python based User Tools which lower the barrier to scientific discovery with DKIST data. The User Tools will enable open, searchable, and documented access to both data and metadata. Utilizing the Advance Scientific Data Format (ASDF) together with the User Tools, users will for example, be able to access all the metadata associated with a given dataset, without having to download the actual data. Mumford et al., describe the User Tools that have been implemented in a poster at this meeting. In addition to the User Tools, data will be accessible a dedicated and richly featured web-based portal. In both the portal and the User Tools, users will have access to browse movies and quality reports, designed to make data selection and filtering a seamless process.
The DC team has created a flexible technological solution that allows it to perform the curation, calibration, and distribution required to support DKIST data and support the work described above. This poster summarizes the work that has been done to create a next generation data facility to support the data from DKIST.