Hi, team! As I have read in previous threads, I've started evaluation of Giraph on IB-cluster. So here I want to share my results (in case it will be useful for anybody) and ask for your ideas of further improving of performance characteristics.
Test system: * 8 Nodes, with dual Intel Xeon CPU E5-2630 (6 cores/CPU), 80GB * Infiniband FDR Dual-Port 4x * SUSE 11.2 * jdk1.7.0_51 At the moment I am performing experiment with SimpleShortestPathsComputation test on generated RMAT graph. I attach plot wich shows scalability of Giraph up to 32 workers. As can be seen from the plot up to 8 workers there is almost linear scalability and then (from 8 to 32) speed is not going up. For me it seems strange that using additional cores on nodes wont bring any performance gain to the execution time. Have anybody meet with such behaviour? Next I am going to use threads instead of workers for cores utilization. Also I am going to switch to the Hadoop-RDMA project. If anybody has any suggestion how I can achieve maximum performance on Giraph on the cluster, I will be obliged to you ;-) Hope for your feedback. Best, Alex
<<attachment: Giraph.png>>
