On Feb 2, 2010, at 16:14 , yogeshwar sonawane wrote: > Hi all, > > Following question can be of naïve nature. Kindly, help me to clarify > the points. > Thanks in Advance. > > [snip]. > > It will be helpful if some information is shared as how mvapich2 > handles the situation regarding above point (while using rdma > operations) ?
Snip from the InfiniBandTM Architecture Specification: o9-20: An application shall not depend of the contents of an RDMA WRITE buffer at the responder until one of the following has occurred: 19 20 21 • Arrival and Completion of the last RDMA WRITE request packet when used with Immediate data. • Arrival and completion of a subsequent SEND message. • Update of a memory element by a subsequent ATOMIC operation. Now, most MPI implementation does not obey this; IIRC MVAPICH does rely on the last byte (word?) being written after all other payload, Scali MPI relies on 4-byte write atomicy and uses a special encoding/decoding which sustains any reordering of the payload (as long as each 4-byte word is written as an atomic unit). The benefit, besides robustness, is the capability of a worm-hole message consumption (the message can be decoded and copied into the MPI user buffer _before_ it has been fully received). This method is currently a patent application, so I do not recommend you to copy-cat this method. So your choice is to be a "nice guy" using one of the preferred methods from the IB standard or rely on how different IB _implementations_ work. Håkon > > I thought that i should mail to both the mailing lists as topic covers > both domains. > > Thanks and Regards, > Yogeshwar > -- > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in > the body of a message to [email protected] > More majordomo info at http://vger.kernel.org/majordomo-info.html Håkon Bugge [email protected] +47 924 84 514 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
