Event Formation Unit - a unified data acquisition pipeline at ESS

M.J. Christensen, T. Bukovics, M. Ipsen, M. Christiansen

The European Spallation Source (ESS) will initially consist of 15 scientific
instruments. These cover diverse scientific scopes—such as neutron spectroscopy,
imaging, diffraction, and reflectometry—each requiring completely different
instrument designs and specialized detector technologies.

The Experiment Control and Data Collection (ECDC) group has developed a unified
data acquisition pipeline to support this heterogeneous system. Within our
readout system, raw analog signals are first processed by digitizers alongside
precise timestamping. The core computational engine of our pipeline is the Event
Formation Unit (EFU). The EFU consists of base software and shared algorithmic
libraries common to all instruments, as well as detector-specific code. This
approach significantly reduces maintenance overhead, ensures consistent logging
and reporting, and enables centralized monitoring.

The EFU is a high-performance C++ application whose primary esponsibilities are
to receive the digitized readout data over UDP, extract and reconstruct distinct
"neutron events", and stream these to a distributed network storage (Apache
Kafka) using Google Flatbuffers for serialisation. These reconstructed events
are fundamentally defined by a relative timestamp and a pixel ID representing
the detector position. Kafka serves as a robust buffer, distributing the live
data to downstream clients, including instrument control systems, live
visualization tools, and the file writer mechanism that produces the final NeXus
scientific data files. This empowers researchers to fully leverage the data
capabilities of ESS.