"Kanevsky, Arkady" <[EMAIL PROTECTED]> writes: > For kDAPL: > The iSER has been submitted to Open Ib by Voltaire already. > NFS-RDMA is at http://sourceforge.net/projects/nfs-rdma/. > > For uDAPL: Oracle, DB2 and MPI. > I am not aware if there is an open source MPI version on uDAPL. > > These are publicly known. > > As far as changing the uDAPL or kDAPL APIs. > There are application already writen to them.
uDAPL at the API level is safe, it is in user space and it is not a kernel concern. Looking at just the API header files kDAPL is a non-starter for kernel inclusion. > There are implementation of these APIs on other platforms besides Linux. > It is in nobody's interest to splinter the user community. So why are these not sane extensions to the standard sockets APIs? I do agree it does not make sense to further fragment things. > We need the same API on all platforms. In the kernel not a chance, that doesn't even make sense. > If there is a good technical reason to change some specific APIs we > should consider it. > But the "burn the spec" approach is not a rationale one. Try do a sensible implementation, instead of burn the spec.... > As far as other transport. As people already mentioned iWARP (IETF > RDDP). Just skimming the DAPL docs DAPL does not yet appear to service anything except IB. > It is still no ready so we will start with gen2. > But lets not loose site of what DAPL brings: > OS independent, > Transport independent, > RDMA APIs!!! At user level that is probably great, if the abstraction layer is not too heavy. Except that MPI is so heavy I would wonder what the point of doing all that when MPI already provides it. Inside an OS independence is nonsense. Sorry kDAPL. ...... With that said there is a fundamental issue with network packet reception in IP. A packet first must be DMA'd to memory to be examined and then copied to it's user for a total of 3 copies through the memory bus to get anywhere. How to solve that problem while still being robust against denial of service attacks, and the other vagaries of hostile public networks and not going the insane path of TCP offload is a challenge. For dealing with very fast networks the number of memory copies start becoming a fundamental limit on how quickly things can go. Currently 4x IB runs at about a 1/3 of the memory bandwidth of a modern memory controller, when you look at it bidirectionally. As network interfaces continue their exponential increase in speed this will only get worse. The various flavors of RDMA seem to be a serious stab at fixing the memory copy problem. How well IP/RDMA or IB/RDMA will actually fix the general problem I don't know, but it certainly deserves a look, and probably a serious kernel design discussion. Beyond that from a practical user level the gen2 stack is very much easier to use than gen1. But gen2 still needs to be extended so user space can do good implementations of uDAPL of MPI. Getting RDMA kernel support or just full featured MPI support to user space from the Linux kernel is going to be a challenge. To get into the mainstream kernel requires a quality work, especially when thing escalate from a single drivers private hack to the next generation version of the sockets API. Kernel maintainers can learn and can be convinced of new things but the developers have to be willing to do the same. This is where running the gauntlet, the real flame fest begins. Good Luck, Eric _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
