AI and (Traditional) HPC Will not Diverge but Rather Converge!

The ascent of Artificial Intelligence is reshaping the high-performance computing (HPC) landscape, prompting questions about a potential crisis for traditional HPC. This talk argues that this perceived divergence is a myth, with an inevitable and opportunistic convergence underway. Market forces are increasingly dictating hardware specifications for AI, compelling HPC to adapt by adopting AI technologies for continued advancement, presenting a significant opportunity for performance leaps in scientific applications. Future systems like FugakuNEXT aim for substantial HPC performance gains and massive AI capabilities by embracing this convergence. A cornerstone of this integration is leveraging low-precision arithmetic, prominent in AI accelerators. Innovations like the Ozaki Scheme II enable ultra-fast emulation of FP64 matrix operations using INT8 Tensor Cores, potentially outperforming native FP64 and allowing a 200 INT8 ExaOps AI machine to achieve an effective 10 Exaflops. This approach shows promise in quantum chemistry and finite-element wave simulations, where INT8 tensor cores have yielded significant speedups while maintaining FP64-equivalent accuracy. The benefits of low precision extend to reducing memory traffic in memory-bound codes, while tools like RAPTOR aid in numerical profiling for safe precision reduction, and AI-driven frameworks like TADASHI automate code transformations with correctness checks. Even interconnect architectures are evolving, with modern AI-driven scale-up and scale-out network designs proving beneficial for diverse HPC workloads, challenging traditional notions. This synergy of AI and HPC, through low-precision computing, emulation, and advanced software, propels us towards Zettascale simulation capabilities within practical power limits, transforming the future of scientific discovery.