Re: [OMPI users] MPI and C++ - now Send and Receive of Classes and STL containers
Hi, On Mon, Jul 06, 2009 at 03:24:07PM -0400, Luis Vitorio Cargnini wrote: > Thanks, but I really do not want to use Boost. > Is easier ? certainly is, but I want to make it using only MPI > itself > and not been dependent of a Library, or templates like the majority > of > boost a huge set of templates and wrappers for different libraries, > implemented in C, supplying a wrapper for C++. > I admit Boost is a valuable tool, but in my case, as much > independent I > could be from additional libs, better. > If you do not want to use boost, then I suggest not using nested vectors but just ones that contain PODs as value_type (or even C-arrays). If you insist on using complicated containers you will end up writing your own MPI-C++ abstraction (resulting in a library). This will be a lot of (unnecessary and hard) work. Just my 2 cents. Cheers, Markus
Re: [OMPI users] MPI and C++ - now Send and Receive of Classes and STL containers
Luis Vitorio Cargnini wrote: just one additional and if I have: vector< vector > x; How to use the MPI_Send MPI_Send([0][0], x[0].size(),MPI_DOUBLE, 2, 0, MPI_COMM_WORLD); ? Vitorio, The standard provides no information on where the different parts of the data will be, relative to each other. In specific, there is no reason to believe that the data in the different internally nested doubles will be contiguous. (In fact, I know of no platform where it will be.) That means trying to send the whole structure at once is problematic. What you wrote will provide a pointer to the first element of the first nested vector to MPI_Send, and the length of that nested vector. If that is what you intend, I expect it to work. (I have not tested it, so I may be misthinking something here.) The other nested vectors could be sent for themselves, using separate MPI_Send calls. The only reliable way to send all of the data at once would be to serialize it off to a single vector or array for the send, then repack it in the structure after it is received. John
Re: [OMPI users] MPI and C++ - now Send and Receive of Classes and STL containers
Thanks, but I really do not want to use Boost. Is easier ? certainly is, but I want to make it using only MPI itself and not been dependent of a Library, or templates like the majority of boost a huge set of templates and wrappers for different libraries, implemented in C, supplying a wrapper for C++. I admit Boost is a valuable tool, but in my case, as much independent I could be from additional libs, better. Le 09-07-06 à 04:49, Number Cruncher a écrit : I strongly suggest you take a look at boost::mpi, http://www.boost.org/doc/libs/1_39_0/doc/html/mpi.html It handles serialization transparently and has some great natural extensions to the MPI C interface for C++, e.g. bool global = all_reduce(comm, local, logical_and()); This sets "global" to "local_0 && local_1 && ... && local_N-1" Luis Vitorio Cargnini wrote: Thank you very much John, the explanation of [0], was the kind of think that I was looking for, thank you very much. This kind of approach solves my problems. Le 09-07-05 à 22:20, John Phillips a écrit : Luis Vitorio Cargnini wrote: Hi, So, after some explanation I start to use the bindings of C inside my C++ code, then comme my new doubt: How to send a object through Send and Recv of MPI ? Because the types are CHAR, int, double, long double, you got. Someone have any suggestion ? Thanks. Vitorio. Vitorio, If you are sending collections of built in data types (ints, doubles, that sort of thing), then it may be easy, and it isn't awful. You want the data in a single stretch of continuous memory. If you are using an STL vector, this is already true. If you are using some other container, then no guarantees are provided for whether the memory is continuous. Imagine you are using a vector, and you know the number of entries in that vector. You want to send that vector to processor 2 on the world communicator with tag 0. Then, the code snippet would be; std::vector v; ... code that fills v with something ... int send_error; send_error = MPI_Send([0], v.size(), MPI_DOUBLE, 2, 0, MPI_COMM_WORLD); The [0] part provides a pointer to the first member of the array that holds the data for the vector. If you know how long it will be, you could use that constant instead of using the v.size() function. Knowing the length also simplifies the send, since the remote process also knows the length and doesn't need a separate send to provide that information. It is also possible to provide a pointer to the start of storage for the character array that makes up a string. Both of these legacy friendly interfaces are part of the standard, and should be available on any reasonable implementation of the STL. If you are using a container that is not held in continuous memory, and the data is all of a single built in data type, then you need to first serialize the data into a block of continuous memory before sending it. (If the data block is large, then you may actually have to divide it into pieces and send them separately.) If the data is not a block of all a single built in type, (It may include several built in types, or it may be a custom data class with complex internal structure, for example.) then the serialization problem gets harder. In this case, look at the MPI provided facilities for dealing with complex data types and compare to the boost provided facilities. There is an initial learning curve for the boost facilities, but in the long run it may provide a substantial development time savings if you need to transmit and receive several complex types. In most cases, the run time cost is small for using the boost facilities. (according to the tests run during library development and documented with the library) John Phillips ___ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users ___ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users ___ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users smime.p7s Description: S/MIME cryptographic signature PGP.sig Description: Ceci est une signature électronique PGP
Re: [OMPI users] MPI and C++ - now Send and Receive of Classes and STL containers
just one additional and if I have: vector< vector > x; How to use the MPI_Send MPI_Send([0][0], x[0].size(),MPI_DOUBLE, 2, 0, MPI_COMM_WORLD); ? Le 09-07-05 à 22:20, John Phillips a écrit : Luis Vitorio Cargnini wrote: Hi, So, after some explanation I start to use the bindings of C inside my C++ code, then comme my new doubt: How to send a object through Send and Recv of MPI ? Because the types are CHAR, int, double, long double, you got. Someone have any suggestion ? Thanks. Vitorio. Vitorio, If you are sending collections of built in data types (ints, doubles, that sort of thing), then it may be easy, and it isn't awful. You want the data in a single stretch of continuous memory. If you are using an STL vector, this is already true. If you are using some other container, then no guarantees are provided for whether the memory is continuous. Imagine you are using a vector, and you know the number of entries in that vector. You want to send that vector to processor 2 on the world communicator with tag 0. Then, the code snippet would be; std::vector v; ... code that fills v with something ... int send_error; send_error = MPI_Send([0], v.size(), MPI_DOUBLE, 2, 0, MPI_COMM_WORLD); The [0] part provides a pointer to the first member of the array that holds the data for the vector. If you know how long it will be, you could use that constant instead of using the v.size() function. Knowing the length also simplifies the send, since the remote process also knows the length and doesn't need a separate send to provide that information. It is also possible to provide a pointer to the start of storage for the character array that makes up a string. Both of these legacy friendly interfaces are part of the standard, and should be available on any reasonable implementation of the STL. If you are using a container that is not held in continuous memory, and the data is all of a single built in data type, then you need to first serialize the data into a block of continuous memory before sending it. (If the data block is large, then you may actually have to divide it into pieces and send them separately.) If the data is not a block of all a single built in type, (It may include several built in types, or it may be a custom data class with complex internal structure, for example.) then the serialization problem gets harder. In this case, look at the MPI provided facilities for dealing with complex data types and compare to the boost provided facilities. There is an initial learning curve for the boost facilities, but in the long run it may provide a substantial development time savings if you need to transmit and receive several complex types. In most cases, the run time cost is small for using the boost facilities. (according to the tests run during library development and documented with the library) John Phillips ___ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users smime.p7s Description: S/MIME cryptographic signature PGP.sig Description: Ceci est une signature électronique PGP
Re: [OMPI users] MPI and C++ - now Send and Receive of Classes and STL containers
I strongly suggest you take a look at boost::mpi, http://www.boost.org/doc/libs/1_39_0/doc/html/mpi.html It handles serialization transparently and has some great natural extensions to the MPI C interface for C++, e.g. bool global = all_reduce(comm, local, logical_and()); This sets "global" to "local_0 && local_1 && ... && local_N-1" Luis Vitorio Cargnini wrote: Thank you very much John, the explanation of [0], was the kind of think that I was looking for, thank you very much. This kind of approach solves my problems. Le 09-07-05 à 22:20, John Phillips a écrit : Luis Vitorio Cargnini wrote: Hi, So, after some explanation I start to use the bindings of C inside my C++ code, then comme my new doubt: How to send a object through Send and Recv of MPI ? Because the types are CHAR, int, double, long double, you got. Someone have any suggestion ? Thanks. Vitorio. Vitorio, If you are sending collections of built in data types (ints, doubles, that sort of thing), then it may be easy, and it isn't awful. You want the data in a single stretch of continuous memory. If you are using an STL vector, this is already true. If you are using some other container, then no guarantees are provided for whether the memory is continuous. Imagine you are using a vector, and you know the number of entries in that vector. You want to send that vector to processor 2 on the world communicator with tag 0. Then, the code snippet would be; std::vector v; ... code that fills v with something ... int send_error; send_error = MPI_Send([0], v.size(), MPI_DOUBLE, 2, 0, MPI_COMM_WORLD); The [0] part provides a pointer to the first member of the array that holds the data for the vector. If you know how long it will be, you could use that constant instead of using the v.size() function. Knowing the length also simplifies the send, since the remote process also knows the length and doesn't need a separate send to provide that information. It is also possible to provide a pointer to the start of storage for the character array that makes up a string. Both of these legacy friendly interfaces are part of the standard, and should be available on any reasonable implementation of the STL. If you are using a container that is not held in continuous memory, and the data is all of a single built in data type, then you need to first serialize the data into a block of continuous memory before sending it. (If the data block is large, then you may actually have to divide it into pieces and send them separately.) If the data is not a block of all a single built in type, (It may include several built in types, or it may be a custom data class with complex internal structure, for example.) then the serialization problem gets harder. In this case, look at the MPI provided facilities for dealing with complex data types and compare to the boost provided facilities. There is an initial learning curve for the boost facilities, but in the long run it may provide a substantial development time savings if you need to transmit and receive several complex types. In most cases, the run time cost is small for using the boost facilities. (according to the tests run during library development and documented with the library) John Phillips ___ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users ___ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users
Re: [OMPI users] MPI and C++ - now Send and Receive of Classes and STL containers
Regardless of MPI, when sending C++ object over the network you have to serialize their contents. The structures, or classes, have to be coded to a stream of bytes, sent over the network, then recoded into their complex object types by the receiving application. There is no way to send object instances in their original memory format because the object layout is dependent on the machine/memory/compiler (plus a number of other things, I'm simplifying here). boost offers a library to easy the serialization work, but you still have to provide hooks to convert the object to a network format. http://www.boost.org/doc/libs/1_39_0/libs/serialization/doc/index.html On Jul 5, 2009, at 8:54 PM, Luis Vitorio Cargnini wrote: Hi, So, after some explanation I start to use the bindings of C inside my C++ code, then comme my new doubt: How to send a object through Send and Recv of MPI ? Because the types are CHAR, int, double, long double, you got. Someone have any suggestion ? Thanks. Vitorio.___ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users
[OMPI users] MPI and C++ - now Send and Receive of Classes and STL containers
Hi, So, after some explanation I start to use the bindings of C inside my C ++ code, then comme my new doubt: How to send a object through Send and Recv of MPI ? Because the types are CHAR, int, double, long double, you got. Someone have any suggestion ? Thanks. Vitorio. smime.p7s Description: S/MIME cryptographic signature PGP.sig Description: Ceci est une signature électronique PGP
