On Mon, Apr 04, 2005 at 07:06:53AM -0700, Roland Dreier wrote: > A memory region (MR) is a memory translation mapping in the HCA's > context. Usually, we create MRs via a firmware command, which is > prohibitively expensive to do in the data path. However, it is > possible for the driver to write directly into the HCA's context, > bypassing the firmware. This is very cheap, just some posted writes, > and so we can do it in the data path. For example, for AIO, SDP uses > this to map a bunch of random userspace pages into something virtually > contiguous in the HCA's memory map, so that it can be used as for RDMA.
Thanks! I understood about 1/2 of that before. I'd like to read a bit more detail though... > However this shouldn't affect IPoIB in the least since a) it doesn't > do any dynamic memory registration and b) it doesn't call any FMR > functions anyway. *nod* - FMR is clearly a red herring in this case. > This is really odd. I don't see how FMRs could directly change IPoIB > performance, since IPoIB isn't using FMRs, even indirectly. Sorry - I said "FMR" when I should have said r210x release. FMR was just recently committed and I assumed (there's that word again) that was related somehow. My bad. > If SDP is > not loaded, then I don't see how it could be used, but the fact that > you get the same number for SDP and IPoIB really makes me think that > the IPoIB number is really an SDP number. Nope - those really where IPoIB numbers. ... > The one that might make a difference is struct mthca_eqe in > mthca_eq.c. Unfortunately simply removing the packed attribute will > break things on 64 bit archs unless the structure is written slightly > differently. It shouldn't be that difficult, so I should have > something for you to test in a day or two. Ok. I won't be able to test that until next week...I'll note which rev picks that up and make sure to test it seperately. BTW, SDP uses "packed" for a dozen or so structures. I haven't looked at any q-syscollect or pfmon data yet to see where SDP is spending time or if "packed" is part of the code path. But I don't have the impression SDP is CPU bound like IPoIB is. thanks, grant _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
