Speaker
Description
Single-particle imaging (SPI) experiments at European XFEL generate massive volumes of data that require fast and reliable pre-processing for efficient experiment optimization and further downstream analysis.
In this contribution, we present the in-house developed SPI pipeline, which is available to the users of European XFEL. This is deployed in both online and offline settings, and includes automated hit finding, hit-rate estimation, background characterization from empty frames, and estimation of particle size and scattering center using a spheroid-based model.
The online component, which enables real-time feedback during experiments, is implemented in Karabo as a set of extensions to the existing pipeline correcting data generated by custom-built detectors. The offline component is implemented as a standalone Python library, which is also integrated into the offline correction pipeline, allowing processing results to be automatically stored within facility-generated corrected data files, and therefore making them readily accessible for subsequent analysis stages.
Many processing components of the SPI pipeline are based on established community approaches, while additional algorithmic solutions were developed to enhance robustness and stability of certain processing steps. As the pipeline is deeply integrated into European XFEL’s data workflows and tools, a seamless use of processing results in subsequent data analysis and reduction workflows is granted.
The SPI pipeline is adopted by experimental teams, and has received positive feedback from our users. Future developments include the exploitation of the online SPI pipeline for on-the-fly data reduction, so as to reduce storage pressure and improve experiment efficiency.
