Speaker
Description
The increasing demand for open, reproducible, and FAIR-compliant research has made robust data management and persistent identification of data, samples, instruments and processing workflows essential in modern scientific large-scale infrastructures. This work presents the current implementation of dataset publication and DOI minting within SciCat at DESY. Datasets are curated, enriched with metadata, and assigned persistent identifiers to support citation, accessibility, and long-term preservation. The DOI minting workflow enables seamless linkage between datasets and associated publications, promoting transparency and reuse of scientific data.
Complementing this effort, we introduce the BlissData Ingestor, a schema-driven tool designed to bridge the gap between raw experimental data and structured data management pipelines at DESY beamlines. The ingestor processes scan data available in BlissData and applies LinkML-based mappings and validation to ensure semantic consistency and data quality prior to ingestion.
A key capability of the presented tool is its support for live metadata handling in ongoing data acquisitions campaigns streaming and retrospective data curation. Leveraging BlissData and it’s events handling of live scans in near real-time, enables integration with online monitoring workflows, while historical scans can be selectively replayed for reprocessing without repeating experiments. Once validated, the data is automatically ingested into SciCat, making it immediately discoverable and accessible through the scientific data catalogue.
By integrating schema-driven validation, automated ingestion, and DOI minting, this work advances towards a more scalable, efficient, and user-friendly data management ecosystem. Together, these developments reduce manual intervention, improve data reliability, and strengthen the overall research data lifecycle from acquisition to publication.
