> Basic ping pong is 25 us. That's fine as this is not a particularly > optimal way to communicate. Each additional server adds 6 us. That > seems like a lot of overhead just to do another pair of posts and > polls, but not my major complaint. Look at the jump from 6 to 7 > servers, 41 us. Beyond that, too. And the standard deviation > becomes huge. A plot of the individual values shows a large spread, > not just a few outliers.
> The hardware is all Mellanox MT25204 I would guess you are seeing the effect of exceeding the size of some internal HCA cache, maybe the QP state cache. But I don't know enough details of the HCA internals to know if this is true and if so which limit you're hitting. - R. _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
