Leveraging Task-Based Programming for Machine Learning and Distributed Training

PyCOMPSs is a task-based programming model for distributed computing. In the recent years, the BSC has been extending this environment for the convergence of HPC + AI workflows. Another development has been the dislib, a distributed machine learning library that implements multiple classical machine learning libraries. While the dislib does not implement neural networks, recently we have been exte
nding it to support distributed training of neural networks with three different synchronization approaches. The talk will describe these developments as well as present some results based on real use cases.