Streamlining HPC Data Workflows: Cloud SQL Databases at Scale for Metadata
Tuesday, June 10, 2025 3:00 PM to Thursday, June 12, 2025 4:00 PM · 2 days 1 hr. (Europe/Berlin)
Foyer D-G - 2nd floor
Women in HPC Poster
Performance and Resource ModelingPerformance Measurement
Information
Poster is on display and will be presented at the poster pitch session.
HPC applications are increasingly generating and using large amounts of data which makes efficient data management extremely essential. As it might be known, relational databases are rarely used in HPC applications since researchers prefer NoSQL databases for their flexibility. However, SQL-based databases on the cloud are being increasingly used for storing and processing metadata in HPC workflows. Nevertheless, their performance has not been extensively benchmarked against HPC workloads in terms of I/O operations, latency, and scalability. This study aims at bridging the gap by benchmarking PostgreSQL performance in a cloud environment to understand its suitability for HPC workloads. Our study investigates the latency, the concurrency, and the scalability. This research aims to improve data management strategies and enhance the overall efficiency of cloud-based databases in the HPC domain.
HPC applications are increasingly generating and using large amounts of data which makes efficient data management extremely essential. As it might be known, relational databases are rarely used in HPC applications since researchers prefer NoSQL databases for their flexibility. However, SQL-based databases on the cloud are being increasingly used for storing and processing metadata in HPC workflows. Nevertheless, their performance has not been extensively benchmarked against HPC workloads in terms of I/O operations, latency, and scalability. This study aims at bridging the gap by benchmarking PostgreSQL performance in a cloud environment to understand its suitability for HPC workloads. Our study investigates the latency, the concurrency, and the scalability. This research aims to improve data management strategies and enhance the overall efficiency of cloud-based databases in the HPC domain.
Format
On DemandOn Site
