Hi Jeff,
thanks for the material!
Meanwhile, I tested sending INT_MAX elements
of an int64_t pair and Open MPI seems to be
fine with sending 2^31 * (2 * 8) bytes.
-Alex
-------------------------------------------
Googletest-Testcase:
TEST(MPImaxBufferSize, maxSendBufferSizeInt) {
MPI_Datatype type;
setup_packed_datatype(&type); // int64_t pair
int sendCount = std::numeric_limits<int>::max();
int rank;
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
if (rank == 0) {
packed_type *buffer = new packed_type[sendCount];
for (int i = 0; i < sendCount; ++i) {
buffer[i].x = 1;
buffer[i].y = 1;
}
MPI_Send(buffer, sendCount, type, 1, 0, MPI_COMM_WORLD);
delete[] buffer;
} else if (rank == 1) {
packed_type *buffer = new packed_type[sendCount];
MPI_Status s;
MPI_Probe(0, 0, MPI_COMM_WORLD, &s);
int receiveCount;
MPI_Get_count(&s, type, &receiveCount);
MPI_Recv(buffer, receiveCount, type, 0, 0, MPI_COMM_WORLD,
MPI_STATUS_IGNORE);
EXPECT_EQ(receiveCount, sendCount);
for (int i = 0; i < receiveCount; ++i) {
EXPECT_EQ(buffer[i].x, 1);
EXPECT_EQ(buffer[i].y, 1);
}
delete[] buffer;
}
}
On 05.06.16 19:07, Jeff Hammond wrote:
Check out the BigMPI project for details on this topic.
Some (many?) MPI implementations still have internal limitations that prevent
one from sending more than 2 gigabytes using MPI datatypes. You can use the
BigMPI tests to identify these.
https://github.com/jeffhammond/BigMPI
https://github.com/jeffhammond/BigMPI-paper
Jeff
On Sunday, June 5, 2016, Alexander Droste <alexander.ra.dro...@googlemail.com
<mailto:alexander.ra.dro...@googlemail.com>> wrote:
Hi Gus,
thanks a lot for the intro, that helps.
Best regards,
Alex
On 05.06.16 18:30, Gustavo Correa wrote:
On Jun 5, 2016, at 12:03 PM, Alexander Droste wrote:
Hi everyone,
I'd like to know what the maximum buffer size
for sends/receives is. Besides the count being limited
to INT_MAX, how is the max buffer size limited?
Best regards,
Alex
Hi Alexander
As far as I know, the usual solution to circumvent
this type of large count problem is to declare an MPI user type to hold
a large number of MPI native types (say,
an MPI_Type_Contiguous or MPI_Type_Vector to hold a bunch of floating
point numbers).
https://www.open-mpi.org/doc/v1.8/man3/MPI_Type_contiguous.3.php
Also, an OMPI pro may correct me for saying foolish things on the list,
but AFAIK, not all sends/receives are buffered, and the buffer size is
set by the default eager/rendevous message threshold (or the value that
you set it to be at runtime with OMPI mca parameters). That buffer size
may also vary according to the btl (sm,vader, tcp, openib, etc).
Search for "eager" and "rendevous" on the FAQ:
https://www.open-mpi.org/faq/?category=all
I hope this helps,
Gus Correa
_______________________________________________
users mailing list
us...@open-mpi.org
Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2016/06/29371.php
_______________________________________________
users mailing list
us...@open-mpi.org
Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2016/06/29372.php
--
Jeff Hammond
jeff.scie...@gmail.com <mailto:jeff.scie...@gmail.com>
http://jeffhammond.github.io/
_______________________________________________
users mailing list
us...@open-mpi.org
Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2016/06/29373.php
