To add some information, the error message is found to be caused by the initialization of packed_oarchive, which calls MPI_Alloc_mem:
oa << sendbuf; Setting export MPI_RANKMEMSIZE=100000000 appears to help for our platform mpi installation. Jiaxin 2015-11-19 16:56 GMT+00:00 Jiaxin Han <[email protected]>: > Hi Lorenz, > > Thank you very much! Your code works! But it seems I have to manually > allocate the memory before unpacking the data from the archive? Then if my > data is a vector of vector, does it mean I have to resize every vector as > well? > > Back to the MPI error, sorry I was wrong to say that it is related to > boost.MPI. With the new code passing MPI_PACKED data, I am receiving the > same out of memory error. So it seems my MPI installation has a problem in > the message size for packed data. Thanks to both Lorenz and Alain! > > Jiaxin > > > 2015-11-19 15:56 GMT+00:00 Lorenz Hübschle-Schneider <[email protected]>: > >> Hi, >> >> On 19/11/15 16:07, Jiaxin Han wrote: >> >>> Hi, >>> >>> Could anyone point me to an example of mixing boost serialization with C >>> MPI functions? >>> >> >> you need to send the archive size first if it's not precisely known. Try >> something like this: >> >> mpi::packed_oarchive oa(comm); >> oa << sendbuf; >> auto sendptr = const_cast<void*>(oa.address()); >> // cast to int because MPI uses ints for sizes like it's still 1990 >> int sendsize = static_cast<int>(oa.size()); >> MPI_Send(&sendsize, 1, MPI_INT, 1, 0, comm); >> MPI_Send(sendptr, sendsize, MPI_PACKED, 1, 0, comm); >> >> The receiving side would look similar: >> >> mpi::packed_iarchive ia(comm); >> int recvsize; >> MPI_Recv(&recvsize, 1, MPI_INT, 0, 0, comm, MPI_STATUS_IGNORE); >> ia.resize(recvsize); >> auto recvptr = ia.address(); >> MPI_Recv(recvptr, recvsize, MPI_PACKED, 0, 0, comm, MPI_STATUS_IGNORE); >> >> >> Do I have to make sure buffer is big enough or the oarchive will handle >>> the memory automatically? If the former, what is the correct memory size >>> to hold a vector? I guess it should hold not just vec.data(), but also >>> vec.size(). >>> >> >> It does this automatically for the sending size. On the receiver, you >> have to know how much data to expect and reserve enough memory accordingly. >> This is what the first transmission in my code above is for. >> >> And lastly, oa does not appear to be the correct variable to pass to >>> MPI_Send. Then what should I pass to MPI_Send after creating the archive? >>> >> >> You pass in the pointer and its length -- oa.address() and oa.size() in >> the example above. >> >> I am asking because the boost mpi installation on our server appears to >>> have a limitation on the message size. For example, the attached code >>> reports an error as: >>> >>> terminate called after throwing an instance of >>> >>> 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::mpi::exception> >>> >' >>> what(): MPI_Alloc_mem: MPI_Alloc_mem: Out of "special" (shared) >>> memory >>> MPI Application rank 0 killed before MPI_Finalize() with signal 6 >>> >>> This appears to be a problem only with the boost installation on the >>> server. The code runs correctly on my local machine. >>> >> >> I'm no expert but to me this sounds like an issue with the MPI >> installation on your servers. Note that it's not Boost.MPI that's giving >> you an error - the error is from the C function that it calls. >> >> Cheers, >> Lorenz >> > >
_______________________________________________ Boost-mpi mailing list [email protected] http://lists.boost.org/mailman/listinfo.cgi/boost-mpi
