Workshop on Sustainable Practices for Reproducibility in HPC

Reproducibility is a cornerstone for trustworthy and robust scientific progress.
The High-Performance Computing (HPC) community often faces reproducibility challenges due to complex software stacks, cutting-edge hardware, and costly operations (computations, data transfers, etc.).
Moreover, as Artificial Intelligence (AI) is taking over the scientific process and as HPC is pivoting towards heavy AI workloads, one might wonder how it would affect the future of reproducibility of experiments in HPC.
The community has set up in the last few years Artifact Evaluation (AE) processes in various venues to raise the reproducibility bar.
While these AE processes managed to improve the rate of authors sharing their code and data, the quality of the artifacts varies.
This could be explained by a lack of education on what reproducibility is, a lack of tools offered by the platforms to support reproducibility, inconsistencies in the various venues' guidelines for packaging the artifacts, or a lack of incentives for the authors to make an extra effort.
Overall, reproducibility in HPC is mostly a methodological and technical problem, which can only be addressed by gathering the community and discussing all together about the way forward.
However, there has not been a regular reproducibility-centric workshop in major HPC venues.

This workshop brings together the HPC community (researchers, practitioners, platform providers, and educators) to share their feedback, tools, and best practices to tackle the reproducibility hurdles met in HPC.

We will explore real-world feedback and lessons learned from artifact authors, reviewers, and chairs, to discuss as a community on innovative ways to reduce the computational and human costs of evaluating reproducibility in HPC.

This workshop will also invite participants to contribute to a community-driven manifesto on sustainable practices for reproducibility in HPC.

---

Organizers:

• Quentin Guilloteau
• Valérie Hayot-Sasson
• Dennis Hoppe
• Josef Weidendorfer