Some penalty, but the largest performance penalty in my case is by far the transmission time. I can serialize and deserialize the data dozens of times before I catch up to transmission penalties.
On Sun, Feb 23, 2014 at 1:58 PM, Matthias Troyer <[email protected]>wrote: > Hi Walt, > > You are aware that you are already incurring a significant performance > penalty by sending objects that need to be packed into an archive and > cannot be sent directly? > > Matthias > > On 23 Feb 2014, at 22:50, Walter Woods <[email protected]> wrote: > > Unless there is a significant performance penalty in each > boost::mpi::communicator holding onto a shadow version of itself, I think > the headaches this would save would be worth it. Simply put, the current > behavior is just very annoying to debug, and it is a somewhat common use > case (especially in applications that know they are receiving exactly N > messages, and want to wait on them simultaneously). > > Thanks, > Walt > > > On Sun, Feb 23, 2014 at 8:18 AM, Matthias Troyer <[email protected]>wrote: > >> Indeed, that is the problem. If we don't want to reserve certain tags for >> internal use of Boost.MPI then the only secure way of solving this problem >> is to create a copy of the communicator, and send the actual message using >> a unique tag in this shadow communicator. We so far hesitated to implement >> this procedure, thinking it to be very unlikely that a user would send a >> second message with the same tag before the first one is received. if this >> should turn out to be a common usage case then we can consider the solution >> I outlined. Does anyone see problems with that solution? >> >> Matthias >> >> >> >> On 23 Feb 2014, at 04:39, Walter Woods <[email protected]> wrote: >> >> > seems to indicate that MPI guarantees that sends and recvs are kept ordered >> on a single-threaded process not using MPI_ANY_SOURCE. If that is the case >> then boost::mpi should as well. >> >> Right, so they are ordered, and that's the problem. >> >> boost::mpi needs to know exactly the size of data that it's receiving. >> So, if you If you're sending / receiving a non-native type, boost::mpi >> needs to transmit how big that data is going to be. Then, it sends the >> data. So one send becomes two sends to MPI - these are ordered. >> >> Receiving is the opposite - it uses one receive to get the size, and then >> *after >> it has the size*, issues another receive to get the data. If you issue >> one irecv command before another has gotten its length (and thus issued its >> data irecv command internally), then because of message ordering, the first >> irecv will get the length, as expected, but then the second irecv will get >> the first's data, mistaking it for a length submission. >> >> Hopefully that makes sense. It's an interleaving problem - because >> everything is ordered, but irecvs turn into two underlying MPI irecvs, the >> two boost::mpi irecvs interleave, causing the problem. >> >> >> On Fri, Feb 21, 2014 at 5:52 PM, Roy Hashimoto >> <[email protected]>wrote: >> >>> On Fri, Feb 21, 2014 at 11:49 AM, Walter Woods <[email protected]>wrote: >>> >>>> In Roy's case, especially the test file, the problem is having multiple >>>> irecv's happening. Lookat the underlying request::handle_serialized_irecv >>>> implementation in boost/mpi/communicator.hpp - one recv is accomplished >>>> through several MPI_IRecv requests issued in sequence. If you have several >>>> irecvs running at once, then one is likely to get the other's data as its >>>> length. >>>> >>> >>> Thanks for your reply and looking at the boost::mpi source - I haven't >>> got that far. I understand what you're saying, but the first few paragraphs >>> of this page: >>> >>> http://www.mpi-forum.org/docs/mpi-1.1/mpi-11-html/node41.html >>> >>> seems to indicate that MPI guarantees that sends and recvs are kept >>> ordered on a single-threaded process not using MPI_ANY_SOURCE. If that is >>> the case then boost::mpi should as well. >>> >>> >>>> In other words, if you want to receive multiple messages in the same >>>> tag, be sure to only have one IRecv() with that tag running at a time. >>>> Data may only be transferred serially (not in parallel) over a single tag >>>> anyhow. >>>> >>> >>> I did change my development code to do this. >>> >>> Hope that helps, >>>> >>> Walt >>>> >>> >>> It does, thanks! >>> >>> Roy >>> >>> _______________________________________________ >>> Boost-mpi mailing list >>> [email protected] >>> http://lists.boost.org/mailman/listinfo.cgi/boost-mpi >>> >>> >> _______________________________________________ >> Boost-mpi mailing list >> [email protected] >> http://lists.boost.org/mailman/listinfo.cgi/boost-mpi >> >> >> >> _______________________________________________ >> Boost-mpi mailing list >> [email protected] >> http://lists.boost.org/mailman/listinfo.cgi/boost-mpi >> >> > _______________________________________________ > Boost-mpi mailing list > [email protected] > http://lists.boost.org/mailman/listinfo.cgi/boost-mpi > > > > _______________________________________________ > Boost-mpi mailing list > [email protected] > http://lists.boost.org/mailman/listinfo.cgi/boost-mpi > >
_______________________________________________ Boost-mpi mailing list [email protected] http://lists.boost.org/mailman/listinfo.cgi/boost-mpi
