Using Transfer Learning for Predicting I/O Time Across Systems
Tuesday, June 10, 2025 3:00 PM to Thursday, June 12, 2025 4:00 PM · 2 days 1 hr. (Europe/Berlin)
Foyer D-G - 2nd floor
Research Poster
High-Performance Data AnalyticsML Systems and ToolsParallel File SystemsPerformance and Resource Modeling
Information
Poster is on display and will be presented at the poster pitch session.
High-quality I/O predictions can improve job scheduling, hardware decisions, and performance optimization. However, creating such predictions is challenging due to complex I/O stack interactions and shared hardware variability. Traditional machine learning requires extensive data and resources, often unavailable to smaller data centers.
We build on Povaliaiev et al.'s transfer learning approach to predict I/O bandwidth with less than 10% of the typical data. We simplify the model by excluding dominant time-related features while maintaining effective I/O time predictions. We managed to create a prediction for applications' I/O time from the Theta cluster using the data from the Blue Waters cluster with around 20% Median Absolute Error, an acceptable quality according to the domain scientists in the field.
Contributors:
High-quality I/O predictions can improve job scheduling, hardware decisions, and performance optimization. However, creating such predictions is challenging due to complex I/O stack interactions and shared hardware variability. Traditional machine learning requires extensive data and resources, often unavailable to smaller data centers.
We build on Povaliaiev et al.'s transfer learning approach to predict I/O bandwidth with less than 10% of the typical data. We simplify the model by excluding dominant time-related features while maintaining effective I/O time predictions. We managed to create a prediction for applications' I/O time from the Theta cluster using the data from the Blue Waters cluster with around 20% Median Absolute Error, an acceptable quality according to the domain scientists in the field.
Contributors:
Format
On DemandOn Site