>From my understanding of the Open MPI code, it uses negative tags to implement >collective operations and one-sided communication only. Your code might work >if you use neither of them but will not be portable, not only across MPI >implementations, but also some future change in Open MPI might break it.
Just my €0.02. Kind regards, Hristo Iliev -- Hristo Iliev, Ph.D. -- High Performance Computing RWTH Aachen University, Center for Computing and Communication Rechen- und Kommunikationszentrum der RWTH Aachen Seffenter Weg 23, D 52074 Aachen (Germany) > -----Original Message----- > From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] > On Behalf Of Sébastien Boisvert > Sent: Friday, September 28, 2012 3:09 PM > To: us...@open-mpi.org > Subject: Re: [OMPI users] About MPI_TAG_UB > > Hello, > > My application has 191 MPI tags allocated with allocateMessageTagHandle, > so > 7 bits is not enough. > > Indeed, tags can be valued from 0 to 2147483647 inclusively with this > MPI_TAG_UB value in Open-MPI. I misused the returned pointer. > > In Open-MPI, MPI_ANY_TAG is -1. I removed the boundary check in > MPI_Isend and MPI_Recv to allow values from -2147483648 to 2147483647 > inclusively. > > https://raw.github.com/sebhtml/patches/master/ompi-1.6.2-ray-4096- > routing.patch > > As long as my tag is not MPI_ANY_TAG, I guess it should work fine although it > is not MPI-compliant. I will test that. > > On 28/09/12 03:50 AM, Iliev, Hristo wrote: > > Hello, > > > > MPI_TAG_UB in Open MPI is INT_MAX == 2^31-1 == 2147483647. The value > > of > > 17438272 (0x10A1640) is a bit strange for MPI_TAG_UB. I would rather > > say that it is the value of a pointer to someplace in the heap, i.e. > > you have missed the fact that the attribute value as returned by > > MPI_Comm_get_attr / MPI_Attr_get is a pointer to the actual value (for > > MPI_TAG_UB it is a pointer to int). > > > > MPI_TAG_UB is predefined attribute and according to §8.1.2 of the MPI > > standard its value cannot be changed by the user. > > > > You have to find another solution, e.g. reduce the tag space to 7 bits > > or put the routing info inside the message payload. > > > > Best regards, > > Hristo Iliev > > -- > > Hristo Iliev, Ph.D. -- High Performance Computing RWTH Aachen > > University, Center for Computing and Communication > > Rechen- und Kommunikationszentrum der RWTH Aachen Seffenter Weg > 23, D > > 52074 Aachen (Germany) > > > >> -----Original Message----- > >> From: users-boun...@open-mpi.org [mailto:users-bounces@open- > mpi.org] > >> On Behalf Of Sébastien Boisvert > >> Sent: Friday, September 28, 2012 1:22 AM > >> To: us...@open-mpi.org > >> Subject: [OMPI users] About MPI_TAG_UB > >> > >> Hello, > >> > >> I am running Ray (a distributed software in genomics) with Open-MPI > >> on > >> 2048 processes and everything runs fine. Ray has a any-to-any > >> communication pattern. > >> To avoid using too much memory, I implemented a virtual message > router. > >> > >> Without the virtual message router, I get messages like these: > >> > >> > [cp2558][[30209,1],0][connect/btl_openib_connect_oob.c:490:qp_create_ > >> o ne] error creating qp errno says Cannot allocate memory > >> > >> We did some tests on the Cray XE6 on 4096 processing elements (4096 > >> MPI > >> ranks) without the virtual message router and everything runs fine as is. > > So > >> using the virtual message router is not required. > >> > >> The real message tag, the real source and the real destination are > >> stored > > in > >> the MPI tag. I know, this is ugly, but it works. I can not store this > > information > >> in the message buffer because the buffer can be NULL. > >> > >> bits 0 to 7: tag (8 bits, values from 0 to 255, 256 possible values) > >> bits > > 8 to 19: > >> true source (12 bits, values from 0 to 4095, 4096 possible values) > >> bits 20 > > to 31: > >> true destination (12 bits, values from 0 to 4095, 4096 possible > >> values) > >> > >> Without the virtual router, my code is compliant with the fact that > >> MPI_Comm_get_attr(MPI_COMM_WORLD, MPI_TAG_UB,...) is at least > >> 32767 (my tags are <= 255). > >> > >> > >> When I try jobs with 4096 processes with the virtual message router, > >> I get > > the > >> error: > >> > >> MPI_ERR_TAG: invalid tag. > >> > >> Without the virtual message router I get: > >> > >> > [cp2558][[30209,1],0][connect/btl_openib_connect_oob.c:490:qp_create_ > >> o ne] error creating qp errno says Cannot allocate memory > >> > >> With Open-MPI 1.5.4, the upper bound is 17438272 (at least in our build). > >> That explains MPI_ERR_TAG. > >> > >> > >> My 2 questions: > >> > >> 1. Is there a better way to store routing information ? > >> > >> 2. Can I create my own communicator and set its MPI_TAG_UB to > >> whatever I want ? > >> > >> > >> Thanks ! > >> > >> *** > >> Sébastien Boisvert > >> Ph.D. student > >> http://boisvert.info/ > >> _______________________________________________ > >> users mailing list > >> us...@open-mpi.org > >> http://www.open-mpi.org/mailman/listinfo.cgi/users > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users
smime.p7s
Description: S/MIME cryptographic signature
