Automated Provisioning of HPC Clusters in the Cloud: Mobility of On-Premise Clusters to the Cloud
Tuesday, June 10, 2025 3:00 PM to Thursday, June 12, 2025 4:00 PM · 2 days 1 hr. (Europe/Berlin)
Foyer D-G - 2nd floor
Women in HPC Poster
Earth, Climate and Weather ModelingHPC in the Cloud and HPC Containers
Information
Poster is on display and will be presented at the poster pitch session.
High Performance Computing (HPC) clusters in the cloud offer researchers the ability to run complex simulations and computations without being tied to on-premise resources. The SCITAS platform at EPFL focuses on automating the creation of cloud-based HPC clusters adapted to users’ specific needs. This flexibility is important for modern scientific workloads, which require scalable and accessible infrastructure. This poster presents the current SCITAS implementation, which employs standard Infrastructure as Code (IaC) technologies such as Ansible, Terraform, and Packer, to deploy fully functional HPC clusters in the cloud. Users can request HPC cluster creation through a web interface by providing basic parameters, after which a system administrator uses Rundeck to trigger the creation process.
This work will be integrated into the SwissTwins project, which has been created to explore and demonstrate the migration of HPC infrastructure from on-premise to public cloud environments with minimal disruption and no changes to the original cluster configuration. SwissTwins is a collaboration among Swiss institutions: CSCS, EPFL, ETHZ, MeteoSwiss, and PSI. A key part of this project is the vCluster technology developed by CSCS, which uses IaC to define HPC clusters as reusable configurations. This approach simplifies the recreation and deployment of clusters in both on-premise and cloud environments.
Future developments will focus on creating self-service solutions for on-demand cloud creation and combining SCITAS' flexible automated setup with the novel vCluster approach to replicate entire HPC environments in the cloud. The objective of this effort is to improve the accessibility and efficiency of using cloud-based HPC solutions for researchers.
Contributors:
High Performance Computing (HPC) clusters in the cloud offer researchers the ability to run complex simulations and computations without being tied to on-premise resources. The SCITAS platform at EPFL focuses on automating the creation of cloud-based HPC clusters adapted to users’ specific needs. This flexibility is important for modern scientific workloads, which require scalable and accessible infrastructure. This poster presents the current SCITAS implementation, which employs standard Infrastructure as Code (IaC) technologies such as Ansible, Terraform, and Packer, to deploy fully functional HPC clusters in the cloud. Users can request HPC cluster creation through a web interface by providing basic parameters, after which a system administrator uses Rundeck to trigger the creation process.
This work will be integrated into the SwissTwins project, which has been created to explore and demonstrate the migration of HPC infrastructure from on-premise to public cloud environments with minimal disruption and no changes to the original cluster configuration. SwissTwins is a collaboration among Swiss institutions: CSCS, EPFL, ETHZ, MeteoSwiss, and PSI. A key part of this project is the vCluster technology developed by CSCS, which uses IaC to define HPC clusters as reusable configurations. This approach simplifies the recreation and deployment of clusters in both on-premise and cloud environments.
Future developments will focus on creating self-service solutions for on-demand cloud creation and combining SCITAS' flexible automated setup with the novel vCluster approach to replicate entire HPC environments in the cloud. The objective of this effort is to improve the accessibility and efficiency of using cloud-based HPC solutions for researchers.
Contributors:
Format
On DemandOn Site
