Enabling High-Performance Hybrid Parallelism: Partitioned Communication Primitives in MPI 4.0

Abstract:  

Partitioned point-to-point communication primitives in MPI 4.0 provide a performance-oriented mechanism for supporting hybrid parallel programming models. These primitives enable an MPI library to transfer portions of a data buffer while the application produces partial contributions using multiple threads or tasks, or through pipelined execution. These capabilities are particularly important for emerging hybrid and heterogeneous applications where overlapping computation and communication is critical for achieving scalable performance on complex architectures.

This talk introduces the partitioned communication API and presents two implementation approaches: a layered library built on top of MPI-3.1 and an integrated solution within Open MPI that natively supports the MPI 4.0 partitioned communication feature set. We compare these approaches in terms of design tradeoffs, performance characteristics, and implementation complexity.

Experimental results highlight the benefits and limitations of each approach (including trade-offs between library-level flexibility and native implementation efficiency), followed by key lessons learned from developing both implementations. The talk concludes with an overview of ongoing efforts to extend partitioned communication primitives to accelerator-based systems.

Bio:

Dr. Purushotham V. Bangalore is the James R. Cudworth Professor in the Department of Computer Science at the University of Alabama. He serves as Associate Director of the Center for Optimized Modern Parallel Adaptive System Software (COMPASS), a Focused Investigatory Center supported by the Predictive Science Academic Alliance Program (PSAAP).

He recently served as a Program Director in the Office of Advanced Cyberinfrastructure within the Directorate for Computer and Information Science and Engineering at the U.S. National Science Foundation, where he contributed to national cyberinfrastructure efforts, including initiatives such as ACCESS and the NAIRR Pilot.

With more than three decades of experience in high-performance computing, his research focuses on designing higher-level abstractions for parallel programming on heterogeneous architectures and next-generation HPC systems, with an emphasis on performance, portability, and predictive modeling. He has contributed to multiple versions of the MPI standard and currently leads research on advanced communication primitives for GPU based system.