Sean Hefty wrote: > The IB SA does something like this. If a user creates a multicast group > with an MGID of 0, the SA will assign an MGID to the group. That MGID > then somehow needs to be distributed to everyone wanting to join that > group.
That's exactly what I want, as long as it's certain the assigned MGID is not already in use. > I'm not aware of any equivalent functionality for IP multicast groups, > so I'm not sure if it makes sense to try to provide this functionality > through the RDMA CM or expose a raw IB multicast interface. I'd prefer a raw IB interface - like you said, this isn't really analogous to IP, and I'd like to avoid the other non-multicast issue I have with RDMA CM. Also, when I first started looking at IB multicast I was expecting this to be part of the ibverbs interface, not a CM. >>> I would then use ObtainMulticastAddress, then pass the returned address >>> via OOB to all the peers I want to be in that multicast group. All the >>> peers would then use JoinMulticastGroup. > > > In order to use what's there, is there any way that the processes can > create unique addresses to use? Maybe map the server port numbers into > the address? Not sure I understand what you're asking.. addresses to use with what? > Your interface looks good. I just need to think more about the details > of what approach to use. I want to find the balance between being easy > to use, but provide the necessary capabilities. Always good :) > I like the idea that the group is automatically deleted when the last > user leaves, since this matches with the IB implementation. We might > also be able to remove the ObtainMulticastAddress, by letting > JoinMulticastGroup take a wildcard address. Can you take a look at the > kernel interface in ib_multicast.h, and let me know if exposing that to > userspace would work for you? Part of why I defined ReturnMulticastAddress the way I did was because I thought it would be useful to hold on to multicast groups without having any peers joined. These could be kept in a pool for re-use, and have peers join/leave them as needed. The MVAPICH group wrote a paper on a similar idea, where they keep a pool of groups with all peers joined, then any peers not interested in communication when a group is pulled from the pool can pull out. But if the time cost is in the join and not the initial creation, this doesn't solve anything. ib_multicast.h looks good.. lots of functionality packed into very few functions. I don't see any problems with it... yet :) I like the callback on join completion, as opposed to polling somewhere. The comments don't say anything about passing an MGID of 0 in - I assume this functionality will be there. Would I pass an MLID of 0 as well, or do I need to come up with a valid MLID from somewhere? Just to make sure, if I pass in an MGID of 0, an MGID will not only be allocated, but joined as well? Again to be clear, ib_free_multicast() will leave the multicast group in question? Is ib_get_mcmember_rec the interface you mentioned for determining whether a port is already in a multicast group? Just thought of a feature that would be nice. As-is, I have no idea when all peers intending to join a multicast group have done so. What would be nice is some sort of notification mechanism - say the ability to provide a callback that is called each time a peer joins a multicast group. I already know which peers I expect to join, so I can keep a list of which ones have/haven't joined, and mark the multicast group as useable when all the expected peers are joined. Would this be reasonable? The alternative for me would be for each peer to send messages OOB to every other peer in the multicast group when it has successfully joined. Andrew _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
