Proposal and Evaluation of Software-Defined Resource Disaggregation for Containerized AI Workloads
Wednesday, June 24, 2026 3:45 PM to 5:15 PM · 1 hr. 30 min. (Europe/Berlin)
Foyer D-G - 2nd Floor
Research Poster
Composable Disaggregated InfrastructureEmerging Computing TechnologiesHeterogeneous System ArchitecturesHPC in the Cloud and HPC ContainersResource Management and Scheduling
Information
Poster is on display and will be presented at the poster pitch session.
Conventional resource allocation in AI pipelines often leads to significant underutilization due to static binding between CPUs and accelerators. While hardware-level Composable Disaggregated Infrastructure (CDI) offers a solution, its requirement for system reboots lacks the agility needed for dynamic workloads. We propose a software-defined resource disaggregation framework optimized for edge-HPC and desktop-cloud environments. Unlike standard microservice architectures, our system enables scaling at the "functional-block" level—finer-grained than typical services—to address specific bottlenecks within a single application pipeline.
This granularity is achieved through a low-latency data exchange layer built on RDMA over Converged Ethernet (RoCE) and distributed shared memory. Our implementation on a heterogeneous cluster of NVIDIA Jetson (AGX Orin/Xavier) nodes demonstrates that the system can dynamically scale a specific bottlenecked function (e.g., Feature Extraction) to match the input stream's frame rate. In our human re-identification (Re-ID) use case, the system scaled throughput from 15 FPS to a real-time target of 60 FPS. We identified that performance peaks at 5 pods due to network congestion, confirmed via experiments with Congestion Notification settings. This work proves that software-defined disaggregation on low-power accelerators provides a high performance-per-watt alternative to centralized high-end GPU systems for specialized AI tasks.
Contributors:
Conventional resource allocation in AI pipelines often leads to significant underutilization due to static binding between CPUs and accelerators. While hardware-level Composable Disaggregated Infrastructure (CDI) offers a solution, its requirement for system reboots lacks the agility needed for dynamic workloads. We propose a software-defined resource disaggregation framework optimized for edge-HPC and desktop-cloud environments. Unlike standard microservice architectures, our system enables scaling at the "functional-block" level—finer-grained than typical services—to address specific bottlenecks within a single application pipeline.
This granularity is achieved through a low-latency data exchange layer built on RDMA over Converged Ethernet (RoCE) and distributed shared memory. Our implementation on a heterogeneous cluster of NVIDIA Jetson (AGX Orin/Xavier) nodes demonstrates that the system can dynamically scale a specific bottlenecked function (e.g., Feature Extraction) to match the input stream's frame rate. In our human re-identification (Re-ID) use case, the system scaled throughput from 15 FPS to a real-time target of 60 FPS. We identified that performance peaks at 5 pods due to network congestion, confirmed via experiments with Congestion Notification settings. This work proves that software-defined disaggregation on low-power accelerators provides a high performance-per-watt alternative to centralized high-end GPU systems for specialized AI tasks.
Contributors:
Format
on-demandon-site
