On Wed, 2006-08-23 at 09:47 -0700, Caitlin Bestler wrote: > [EMAIL PROTECTED] wrote: > > Quoting r. john t <[EMAIL PROTECTED]>: > >> Subject: basic IB doubt > >> > >> Hi > >> > >> I have a very basic doubt. Suppose Host A is doing RDMA write (say 8 > >> MB) to Host B. When data is copied into Host B's local > > buffer, is it guaranteed that data will be copied starting > > from the first location (first buffer address) to the last > > location (last buffer address)? or it could be in any order? > > > > Once B gets a completion (e.g. of a subsequent send), data in > > its buffer matches that of A, byte for byte. > > An excellent and concise answer. That is exactly what the application > should rely upon, and nothing else. With iWARP this is very explicit, > because portions of the message not only MAY be placed out of > order, they SHOULD be when packets have been re-ordered by the > network. But for *any* RDMA adapter there is no guarantee on > what order the adapter flushes things to host memory or particularly > when old contents that may be cached are invalidated or updated. > The role of the completion is to limit the frequency with which > the RDMA adapter MUST guarantee coherency with application visible > buffers. The completion not only indicates that the entire message > was received, but that it has been entirely delivered to host memory.
Actually, A knows the data is in B's memory when A gets the completion notice. B can't rely on anything unless A uses the RDMA write with immediate which puts a completion event in B's CQ. Most applications on B ignore this requirement and test for the last memory location being modified which usually works but doesn't guarantee that all the data is in memory. _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
