Dynamic Use of Resources for Affordable Exascale and Beyond

The rapid growth in computing demands in science, industry, and academia has led to more and larger systems for HPC and artificial intelligence (AI). This strong increase in computing capacity has also brought significant challenges: high energy consumption, danger of resource underutilization, and questionable environmental sustainability. This BoF will specifically target truly dynamic resource management to improve resource utilization and energy efficiency/carbon footprint.

Breaking out of the curse of static resource allocation, as used today in classical batch scheduling systems, is required for achieving high resource utilization. The old scheme does not consider temporal variations in workload intensity, which leads to over-provisioned and under-used resources. Dynamic resource management frameworks rely on real-time system monitoring, application-provided information, and adaptive/predictive scheduling algorithms to optimize the allocation of compute, memory, and storage resources during runtime. This BoF will discuss the challenges of dynamic resource management and how it can significantly improve utilization rates, reduce energy use and operating costs, and in effect lower the carbon footprint of HPC and AI systems.

Current approaches for energy-efficient HPC emphasize power-aware scheduling, dynamic voltage-frequency scaling (DVFS), and workload consolidation, to name just a few. These existing strategies must be integrated with dynamic resource management, intelligent workload migration, and predictive models based on machine learning, enabling dynamic balancing of power consumption, I/O bandwidth, and compute throughput.

This BoF will address fundamental research challenges such as dynamic scheduling, resource management and control, application co-design, and data co-location. Such research will finally be related to cutting-edge strategies, tools, and methodologies for achieving energy-efficient, green, and optimized use of large-scale HPC resources. The BoF is strongly supported by a panel of internationally leading experts.

Dynamic resource utilization requires changes across the entire HPC software stack. Consequently, this BoF seeks collaborative opportunities to advance research and development in dynamic resource utilization. This includes fostering partnerships between academia, industry, and government to develop scalable open-source tools for dynamic resource management and ensure that the required support will be integrated into important software and standards like MPI, OpenMP, Flux, and AI frameworks. Such results will be an important part of the next generation of sustainable HPC technologies.

---

Organizers:

• Martin Schreiber
• Antonio J. Peña
• Jesus Carretero Perez
• Pierre-François Dutot
• Estela Suarez