The Sky at 1 Terabit Per Second: Challenges and Architecture of the Argus Optical Array Data Pipelines

The Argus Optical Array is a new type of synoptic survey observatory, with a total collecting area equivalent to a 5-meter monolithic telescope and an all-sky field of view multiplexed from 900 commercial-off-the-shelf telescopes. The Argus Optical Array will operate with both a 60-second base cadence and a 1-second fast cadence mode, reaching g=19.6 in 60-seconds and g=21.9 each hour, enabling searches for optical counterparts to fast radio bursts (FRBs) and gravitational wave events, exoplanet microlensing events, occultations by distant solar system bodies, and a variety of other high-speed transient phenomena. We will install the fully-funded, 48-node Argus Pathfinder at the Pisgah Astronomical Research Institute in Q3 2022. Argus Pathfinder will image the entire Northern sky once per night in a series of 60 pointing “ratchets.” Operating in the fast-cadence mode, the Argus Pathfinder alone will produce hundreds of terabytes of data per night at 100 Gbps, an order of magnitude higher than other data-intensive surveys currently operating or planned, but an order of magnitude lower than the 1 Tbps data rate of the full 900-node system. The Argus Array Hierarchical Data Processing System (HDPS) is the control and data reduction system for both Argus Pathfinder and Argus Optical Array. To support the peak 55.8 GPix per second data rate of the full array, Argus-HDPS exploits the natural parallelism of the multiplexed observatory by pairing each optical system with a dedicated compute node able to reduce the raw incoming images into preliminary data products, including second-cadence transient alerts, postage-stamp cutouts for selected targets, and full-frame images at 10-minute cadence. In this poster, I will describe our hierarchical data reduction strategy and pipelines, and discuss their deployment and status for Argus Pathfinder.