Roland Dreier wrote:
OK, I will merge this for 2.6.24. However, I still don't really understand the changelog entry:
The kernel IB stack allows (through the RDMA CM) user space multicast applications to interoperate with IP based apps optionally running at a different IP subnet. To support this inter-op for the case where the receiving party resides at the IB side, there is a need to handle IGMP (reports/queries) else the local IP router would not forward multicast traffic towards the IB network.
So in other words you have a userspace app that joins an IPoIB multicast group and then it has to do an IP_ADD_MEMBERSHIP socket option to trigger IGMP messages being sent out, so that traffic gets routed to it?
yes
This patch does a lookup on the database used for multicast reference counting and enhances IPoIB to ignore multicast group which is already handled by user space, all this under a per device policy flag. That is when the policy flag allows it, IPoIB will not join and attach its QP to a multicast group which has an entry on the database.
And then you don't want the kernel IPoIB driver to actually join the multicast group for the IP multicast group you added with IP_ADD_MEMBERSHIP? Why is that exactly -- this is the part I'm especially hazy on.
Yes I don't want ipoib to receive packets from this group, so it need not join/attach to it through the flow of the net core calling ipoib's set_multicast_list callback. In the case of IGMP v2 where reports are sent over the actual group, IPoIB does join but as "send only", I have validated this to work fine with my patch.
The whole idea is that there's a userspace app that joins through the rdma-cm and attaches its user space QP to this MGID such that it will receive this multicast group packets. Opening a socket and calling add membership on it is done since this is the only means to cause the kernel to issue IGMP reporting etc on this group.
Other than that IPoIB need not join/attach to this group, doing so on my system (*) cuts the performance by half. When I attach two user space processes to the same group performance is cut by only ~10%, so the 50% drop might turn to be network stack issue or firmware issue or combination of both and other things.
At the bottom line, the umcast flag allow users who need to interop with IP routers, to signal IPoIB that they don't want groups joined for user space receiving to be joined/attached by the kernel.
are we done? Or. (*) Mellanox Arbel memfull hca (device 25208), firmware 4.7.600 _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
