On Tue, Oct 06, 2009 at 12:05:02PM -0700, Sean Hefty wrote: > From user space, the call sequence does not change. The user calls > rdma_resolve_addr, rdma_resolve_route, rdma_connect, etc. It is up to the > librdmacm to perform the resolution. Today, the resolution request is simply > passed down to the kernel, which restricts how the resolution can be > performed.
Well, that is the whole trade off for using IP based addressing via RDMA CM - you completely give up the IB addressing architecture. Having an option for user space to specify an IP address and a HW address is just really very, very strange. I understand why you'd want to do this, but it is just such a brutal hack.. IMHO, I think the fundamental problem is that RDMA CM is tied to IP (ie iWarp addressing). This was probably a mistake. We really needed a protocol and address agnostic CM API that can work with RDMA IB, iWARP and standard IB CM protocols, using the appropriate protocol specific addressing family and mechanism in each case. Then you could have ACM provide GID based PR addressing using IB CM within the RDMA CM API and it doesn't create such a complex mess. This would then mirror how sockets work. Actually, thinking about it some more, that would be very helpful. As I said before, I have worked on apps using IB CM. The only reason is to have complete control over the addressing. If I could use RDMA CM API in some kind of AF_GID addressing and service ID space, it would basically eliminate the need for IB CM entirely and make it alot less trouble to support things like iWarp, since it now just another AF/PF in the same API family. > (FYI - I have not yet implemented the librdmacm to call > rdma_bind_addr as part of rdma_resolve_route on linux. I did not > see an easy way to convert a destination IP address to a source IP > address. If anyone knows how, please let me know.) Isn't this related to the now several threads discussing how route resolution in the RDMA CM is busted? http://lists.openfabrics.org/pipermail/general/2009-July/060612.html You get the source address via the user (netlink) or kernel (ip_route_output_key) equivalent of 'ip route get x.x.x.x dev XXX' I wonder if Leo ever figured this out? Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
