> The RDMA library has features for joining a multicast group. Could you > point me at the appropriate specifications about infiniband relating to > the limitations of RDMA when used with multicast (or hand me an > explanation would be fine too :)
RDMA (in the sense of remote direct memory access, ie one sided operations) can only be done over reliable connected transport. Multicast works only over unreliable datagram transport. You can get kernel bypass and zero-copy send/receive for multicast with libibverbs and librdmacm. But that strictly speaking isn't RDMA. > Also to answer your questions about performance requirements: today > totem is entirely cpu bound when running on GIGE hardware with jumbo > frames. A majority of this time I believe is spent copying message data > between kernel and userspace, and then the other copies that take place > around the protocol. I'd like to remove any copies that are present if > possible to give the CPU more bandwidth to execute the protocol. Seems implausible given today's memory bandwidths. I've seen slightly in excess of 10 gigabits/sec with TCP on IPoIB -- this is with netpipe and using large send/large receive offload but still all data is being copied at least once. Profiling to find out where your CPU is really going probably wouldn't be a bad idea before you go to the effort of reimplementing everything on InfiniBand. - R. _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
