On Fri, Apr 01, 2005 at 06:40:48PM -0800, Grant Grundler wrote: > Hi, > Just wanted to share initial perf results (and surprise) > that I'm getting on the HP ZX1/IA64 boxes. > > Before FMR support was committed, netperf was reporting around > 1720 Mb/s (215 MB/s) for IPoIB with msi_x=1 and netserver pinned > to the CPU that wasn't taking interrupts. After FMR was committed, > netperf is reporting about 3500 Mb/s (437 MB/s) for IPoIB. CPU was > saturated on the send side in all cases. > > I've a vague idea what "Fast Memory Registration" is but not a good > understanding. Can someone point me at a decent explanation of FMR?
A few people responded with good FMR descriptions. However, I'd like to add that when using an unmodified version of netperf, neither IPoIB or SDP are using FMRs. IPoIB never uses FMRs, and currently SDP only uses FMRs when the application is using Linux AIO to read or write data on the socket. In that instance if the buffers are larger then a threshold value they will be registered using FMRs and the contiguous address is then shared with the remote half of the connetion which can then RDMA to/from the buffer. The example code (ttcp.aio.c) I checked in will use AIO and FMRs if the transfer size (-l) is over the 5K default threshold. -Libor _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
