A collective communication library plugined into Hadoop from Indiana University

Why Harp?


Speed and Performance
Harp is built with a wide array of architectural improvements that contribute to high efficiency gains.
Compatibility with Hadoop
Harp is a plugin built into Apache Hadoop and hence no code change of Hadoop is required for migration.
Ease of Development and Troubleshooting
Easily debug and identify the issues, allowing faster iteration during development.
Scalability and Portability
Harp is highly scalable in the ability to run on large amount of training data and model parameters. It can run on both cloud and HPC platforms.

 


Examples