Speaker
Description
The Karabo data acquisition (DAQ) at the European XFEL is capable of data reduction based on external information like beam conditions and shutter positions. To cope with the storage challenges imposed by the next generation of the AGIPD detector, i.e. almost 1 GB/s for each of its 56 modules, it will be necessary to base the decision to store or reject data frames on near-realtime data analysis like hit finding for the SFX experiments this detector will be used for. This requires to stream data at the full rate from the C++-based DAQ to the Python-based calibration and data analysis algorithms. The infiniband network infrastructure at the European XFEL offers the required bandwidth, but the TCP/IP-based Karabo data pipelines do not cope reliably enough with the required data rate. To overcome this shortcoming, Karabo data exchange options have been extended with Remote Direct Memory Access (RDMA).
This contribution presents design choices, lessons learned, achieved data
rates, and future development directions of the RDMA communication between the Karabo DAQ, calibration and online analysis.
