Let me add a little more color to William's response. The general theme is: it depends on what the underlying network provides.
Some underlying networks natively support one-sided operations like PUT / WRITE and GET / READ (e.g., IB/RDMA, RoCE/RDMA, ... etc.). Some don't (like TCP). Open MPI will adapt to use whatever transports the underlying network supports. Additionally, the determination of whether Open MPI uses a "two sided" or "one sided" type of network transport operation depends on a bunch of other factors. The most efficient method to get a message from sender to receive may depend on issues such as the size of the message, the memory map of the message, the current network resource utilization, the specific MPI operation, ... etc. Also, be aware that "RDMA" commonly refers to InfiniBand-style one-sided operations. So if you want to use "RDMA", you may need to use an NVIDIA-based network (e.g., IB or RoCE). That's not the only type of network one-sided operations available, but it's common. -- Jeff Squyres jsquy...@cisco.com ________________________________________ From: devel <devel-boun...@lists.open-mpi.org> on behalf of Zhang, William via devel <devel@lists.open-mpi.org> Sent: Wednesday, April 20, 2022 6:12 PM To: Open MPI Developers Cc: Zhang, William Subject: Re: [OMPI devel] RDMA and OMPI implementation Hello Masoud, Responded inline Thanks, William From: devel <devel-boun...@lists.open-mpi.org> on behalf of Masoud Hemmatpour via devel <devel@lists.open-mpi.org> Reply-To: Open MPI Developers <devel@lists.open-mpi.org> Date: Wednesday, April 20, 2022 at 5:29 AM To: Open MPI Developers <devel@lists.open-mpi.org> Cc: Masoud Hemmatpour <mashe...@gmail.com> Subject: [EXTERNAL] [OMPI devel] RDMA and OMPI implementation CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Hello Everyone, Sorry, MPI is quite new for me, in particular the implementation. If you don't mind, I have some very basic questions regarding the OMPI implementation. If I use one-sided MPI operations (Get, and Put) forcefully I use RDMA? – It depends, but it’s not guaranteed. For example, in Open MPI 4.0.x, there was the osc/pt2pt component that implemented osc operations using send/receive. Or for example, with calls to libfabric’s osc api, it depends on the implementation of the underlying provider. Is it possible to have one-sided without RDMA? - Yes In general, other types of MPI operations like Send/Receive or collective operations are implemented using RDMA or not necessarily? – Not necessarily. For example, using TCP won’t use RDMA. The underlying communication protocol could very well implement send/receive using RDMA though. How can I be sure that I am using RDMA for a specific operation? – I’m not sure there’s an easy way to do this, I think you have to have some understanding of what communication protocol you’re using and what that protocol is doing. Thank you very much in advance for your help! Best Regards,
