Speaker
Description
The High Energy Photon Source (HEPS) generates massive amounts of highly heterogeneous experimental data, placing significant pressure on storage, computing, and network resources. Data compression is an effective way to mitigate data growth, but a single compression method cannot meet the diverse requirements in the HEPS context. To address this issue, this work first studies a compression method recommendation framework for different beamline experiments, enabling adaptive selection of compression methods based on data characteristics. Second, for data that are difficult to compress, deep learning techniques are used to further explore the potential of lossless compression. Finally, lossy compression methods for scientific computing scenarios are explored, with different loss quantification designs tailored to different computing tasks. This work provides systematic solutions and key technical support for data compression at HEPS.
