On Tue, 2006-08-15 at 12:39 -0700, Sean Hefty wrote: > >Why are these separated? Isn't an address handle needed for each > >destination QP? If so, then why is the remote qpn/qkey also needed to > >transmit a datagram? > > The address handle doesn't include QPN/QKey information. Maybe think of them > more as specifying the path to some port. >
Ok. >From what I can tell via experimentation, the qkey of the mcast group doesn't need to have any relation to the qkeys of the qps. I was able to create a mcast group with the mc qkey==0xe00a0a0a, and 3 apps joined this group, but their qp qkeys were 0 (I changed ucma_init_ud_qp() to set the qp qkey to 0). One app sent to the mcgroup ah/qkey/qpn and the other two received the packet. Does that make sense? So maybe all we need is the concept of REUSE_PORT to allow multiple librdma users to create cm_ids with the same local port. currently this isn't allowed. If we do this, then all processes that want to exchange mcast packets would create cm_ids and do rdma_resolve_addr() with the same src port number on all systems. Senders send to the ah/remote_qpn/remote_qkey of the mcast group. This routes packets to all IB ports that have subscribers. Then since the sender's qp has the same qkey as all the group participants each qp will receive a copy of the packet. The mcast setup code in librdma doesn't need to change. IE the qkey can remain the ip mcast address. I think this will work. It is similar to UDP/IP/MCAST... Or am I all wet? whatchathink? _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
