Oups, forgot to send it to the list.
=================================
So, here is next patch, I think it integrates all the discussions except
for the throwing ctor, which can be added later if a consensus is
reached.
I did modify the documentation, but I do not know how to test those
changes. They are quite simple, but still...
As for the throwing ctors, my position was that I do not see a strong
argument in their favor, and would like to see what the user are saying
(as a user, they would be of no use, I'd like to get 'multiple', but
will need to deal with the other possibilities anyway, so throwing is
not a option. But that's just a sample of one).
Regards
On Thu, 2013-01-17 at 08:27 -0700, Matthias Troyer wrote:
> Thank you! I've committed the patch to config.hpp to the trunk and have also
> updated the documentation.
>
> Matthias
>
> On Jan 16, 2013, at 4:08 PM, Alain O Miniussi <[email protected]> wrote:
>
> > There is an attached patch with that change.
> > If it's ok, and once commited, I'll move to the thread init.
> >
> >
> > On Mon, 2012-12-31 at 23:05 +0100, Matthias Troyer wrote:
> >> On Dec 4, 2012, at 10:53 AM, Alain O Miniussi <[email protected]>
> >> wrote:
> >>
> >>> On Thu, 2012-11-29 at 12:55 -0500, Jeremiah Willcock wrote:
> >>>> On Thu, 29 Nov 2012, Matthias Troyer wrote:
> >>>>
> >>>>>
> >>>>> On Nov 29, 2012, at 5:23 PM, Jeremiah Willcock <[email protected]>
> >>>>> wrote:
> >>>>>>
> >>>>>> Another issue with MPI versions is that Boost.MPI currently uses
> >>>>>> functions such as MPI_Address that have been removed in MPI 3.0. Is
> >>>>>> that something that should be addressed in the future? I think the
> >>>>>> replacements for some of them did not exist before MPI 2.0.
> >>>>>
> >>>>> A valid point - we will have to provide two implementations depending
> >>>>> on
> >>>>> which version of MPI is present. Is there any standardized macro that
> >>>>> one an check to determine the MPI version at compile time?
> >>>>
> >>>> There are MPI_VERSION and MPI_SUBVERSION, but those are in 2.0 and above
> >>>> only as well. You can probably use their being undefined to mean that
> >>>> the
> >>>> implementation does not comply to 2.0. I don't know how many 1.1-only
> >>>> MPI
> >>>> implementations are around anymore, though; there may not be any still
> >>>> in
> >>>> use (MPI 2.0 was released in 1997).
> >>>
> >>> That's a good point, especially if the "historical" C++ API was not
> >>> present in 1.1 (which I think it's the case, it's not clear to me if it
> >>> was introduced in 1.2 or 2.0). Maybe we should only take into
> >>> consideration 2.0 and 3.0.
> >>> Also, if some 1.1 only API are still use, how many requires a Boost.MPI
> >>> interface ?
> >>>
> >>> As far as the no arg init is concerned, it seems to me that that
> >>> - providing them will simplify the API,
> >>> - even if some 1.1 only MPI are still around, even if some of them have
> >>> a use for Boost.MPI, even if those do not support a no arg Init, theirs
> >>> users still have the possibility to provide those arguments anyway.
> >>>
> >>> Alain
> >>
> >> I checked the implementation and we do have a no arg init for MPI 2.0 or
> >> higher since that can be tested by MPI_VERSION. You can also manually
> >> define BOOST_MPI_HAS_NOARG_INITIALIZATION to enable it for MPI 1.2 or MPI
> >> 1.3 implementations. I assume that this is good enough and we might just
> >> want to edit the documentation to make the no arg version the default in
> >> the documentation.
> >>
> >> Matthias
> >>
> >>
> >
> > --
> > Alain Miniussi
> > Responsable Tech. Centre de Calcul Haute Performance
> > Obs. de la Côte d'Azur |Mont Gros: +33 4 92 00 30 09
> > BP 4229 |Sophia : +33 4 83 61 85 44
> > 06304 Nice Cedex 4 |https://crimson.oca.eu
> > <mpiinit-r82503.patch>_______________________________________________
> > Boost-mpi mailing list
> > [email protected]
> > http://lists.boost.org/mailman/listinfo.cgi/boost-mpi
>
>
--
Alain Miniussi
Responsable Tech. Centre de Calcul Haute Performance
Obs. de la Côte d'Azur |Mont Gros: +33 4 92 00 30 09
BP 4229 |Sophia : +33 4 83 61 85 44
06304 Nice Cedex 4 |https://crimson.oca.eu
Index: boost/mpi/environment.hpp
===================================================================
--- boost/mpi/environment.hpp (révision 82588)
+++ boost/mpi/environment.hpp (copie de travail)
@@ -17,9 +17,46 @@
#include <boost/noncopyable.hpp>
#include <boost/optional.hpp>
#include <string>
+#include <iosfwd>
namespace boost { namespace mpi {
+namespace threading {
+/** @brief specify the supported threading level.
+ *
+ * Based on MPI 2 standard/8.7.3
+ */
+enum level {
+ /** Only one thread will execute.
+ */
+ single = MPI_THREAD_SINGLE,
+ /** Only main thread will do MPI calls.
+ *
+ * The process may be multi-threaded, but only the main
+ * thread will make MPI calls (all MPI calls are ``funneled''
+ * to the main thread).
+ */
+ funneled = MPI_THREAD_FUNNELED,
+ /** Only one thread at the time do MPI calls.
+ *
+ * The process may be multi-threaded, and multiple
+ * threads may make MPI calls, but only one at a time:
+ * MPI calls are not made concurrently from two distinct
+ * threads (all MPI calls are ``serialized'').
+ */
+ serialized = MPI_THREAD_SERIALIZED,
+ /** Multiple thread may do MPI calls.
+ *
+ * Multiple threads may call MPI, with no restrictions.
+ */
+ multiple = MPI_THREAD_MULTIPLE
+};
+/** Formated output for threading level. */
+std::ostream& operator<<(std::ostream& out, level l);
+
+/** Formated input for threading level. */
+std::istream& operator>>(std::istream& in, level& l);
+} // namespace threading
/** @brief Initialize, finalize, and query the MPI environment.
*
* The @c environment class is used to initialize, finalize, and
@@ -62,6 +99,22 @@
* program if it is destructed due to an uncaught exception.
*/
explicit environment(bool abort_on_exception = true);
+ /** Initialize the MPI environment.
+ *
+ * If the MPI environment has not already been initialized,
+ * initializes MPI with a call to @c MPI_Init_thread. Since this
+ * constructor does not take command-line arguments (@c argc and @c
+ * argv), it is only available when the underlying MPI
+ * implementation supports calling @c MPI_Init with @c NULL
+ * arguments, indicated by the macro @c
+ * BOOST_MPI_HAS_NOARG_INITIALIZATION.
+ *
+ * @param mt_level the required level of threading support.
+ *
+ * @param abort_on_exception When true, this object will abort the
+ * program if it is destructed due to an uncaught exception.
+ */
+ explicit environment(threading::level mt_level, bool abort_on_exception = true);
#endif
/** Initialize the MPI environment.
@@ -80,6 +133,25 @@
*/
environment(int& argc, char** &argv, bool abort_on_exception = true);
+ /** Initialize the MPI environment.
+ *
+ * If the MPI environment has not already been initialized,
+ * initializes MPI with a call to @c MPI_Init_thread.
+ *
+ * @param argc The number of arguments provided in @p argv, as
+ * passed into the program's @c main function.
+ *
+ * @param argv The array of argument strings passed to the program
+ * via @c main.
+ *
+ * @param mt_level the required level of threading support
+ *
+ * @param abort_on_exception When true, this object will abort the
+ * program if it is destructed due to an uncaught exception.
+ */
+ environment(int& argc, char** &argv, threading::level mt_level,
+ bool abort_on_exception = true);
+
/** Shuts down the MPI environment.
*
* If this @c environment object was used to initialize the MPI
@@ -185,13 +257,21 @@
*/
static std::string processor_name();
+ /** Query the current level of thread support.
+ */
+ static threading::level thread_level();
+
+ /** Are we in the main thread?
+ */
+ static bool main_thread();
+
private:
/// Whether this environment object called MPI_Init
bool i_initialized;
/// Whether we should abort if the destructor is
bool abort_on_exception;
-
+
/// The number of reserved tags.
static const int num_reserved_tags = 1;
};
Index: libs/mpi/test/mt_init_test.cpp
===================================================================
--- libs/mpi/test/mt_init_test.cpp (révision 0)
+++ libs/mpi/test/mt_init_test.cpp (révision 0)
@@ -0,0 +1,27 @@
+// Copyright (C) 2013 Alain Miniussi <[email protected]>
+
+// Use, modification and distribution is subject to the Boost Software
+// License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at
+// http://www.boost.org/LICENSE_1_0.txt)
+
+// test threading::level operations
+
+#include <boost/mpi/environment.hpp>
+#include <boost/test/minimal.hpp>
+#include <iostream>
+#include <sstream>
+
+namespace mpi = boost::mpi;
+
+int
+test_main(int argc, char* argv[]) {
+ mpi::threading::level required = mpi::threading::level(-1);
+ BOOST_CHECK(argc == 2);
+ std::istringstream cmdline(argv[1]);
+ cmdline >> required;
+ BOOST_CHECK(!cmdline.bad());
+ mpi::environment env(argc,argv,required);
+ BOOST_CHECK(env.thread_level() >= mpi::threading::single);
+ BOOST_CHECK(env.thread_level() <= mpi::threading::multiple);
+ return 0;
+}
Index: libs/mpi/test/mt_level_test.cpp
===================================================================
--- libs/mpi/test/mt_level_test.cpp (révision 0)
+++ libs/mpi/test/mt_level_test.cpp (révision 0)
@@ -0,0 +1,107 @@
+// Copyright (C) 2013 Alain Miniussi <[email protected]>
+
+// Use, modification and distribution is subject to the Boost Software
+// License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at
+// http://www.boost.org/LICENSE_1_0.txt)
+
+// test threading::level operations
+
+#include <boost/mpi/environment.hpp>
+#include <boost/test/minimal.hpp>
+#include <iostream>
+#include <sstream>
+
+namespace mpi = boost::mpi;
+
+void
+test_threading_level_io(mpi::threading::level orig) {
+ std::ostringstream out;
+ namespace mt = boost::mpi::threading;
+ mt::level printed = mt::level(-1);
+
+ out << orig;
+ BOOST_CHECK(out.good());
+ std::string orig_str(out.str());
+ std::cout << "orig string:" << orig_str << '\n';
+ std::istringstream in(orig_str);
+ in >> printed;
+ BOOST_CHECK(!in.bad());
+ std::cout << "orig: " << orig << ", printed: " << printed << std::endl;
+ BOOST_CHECK(orig == printed);
+}
+
+void
+test_threading_levels_io() {
+ namespace mt = boost::mpi::threading;
+ test_threading_level_io(mt::single);
+ test_threading_level_io(mt::funneled);
+ test_threading_level_io(mt::serialized);
+ test_threading_level_io(mt::multiple);
+}
+
+void
+test_threading_level_cmp() {
+ namespace mt = boost::mpi::threading;
+ BOOST_CHECK(mt::single == mt::single);
+ BOOST_CHECK(mt::funneled == mt::funneled);
+ BOOST_CHECK(mt::serialized == mt::serialized);
+ BOOST_CHECK(mt::multiple == mt::multiple);
+
+ BOOST_CHECK(mt::single != mt::funneled);
+ BOOST_CHECK(mt::single != mt::serialized);
+ BOOST_CHECK(mt::single != mt::multiple);
+
+ BOOST_CHECK(mt::funneled != mt::single);
+ BOOST_CHECK(mt::funneled != mt::serialized);
+ BOOST_CHECK(mt::funneled != mt::multiple);
+
+ BOOST_CHECK(mt::serialized != mt::single);
+ BOOST_CHECK(mt::serialized != mt::funneled);
+ BOOST_CHECK(mt::serialized != mt::multiple);
+
+ BOOST_CHECK(mt::multiple != mt::single);
+ BOOST_CHECK(mt::multiple != mt::funneled);
+ BOOST_CHECK(mt::multiple != mt::serialized);
+
+ BOOST_CHECK(mt::single < mt::funneled);
+ BOOST_CHECK(mt::funneled > mt::single);
+ BOOST_CHECK(mt::single < mt::serialized);
+ BOOST_CHECK(mt::serialized > mt::single);
+ BOOST_CHECK(mt::single < mt::multiple);
+ BOOST_CHECK(mt::multiple > mt::single);
+
+ BOOST_CHECK(mt::funneled < mt::serialized);
+ BOOST_CHECK(mt::serialized > mt::funneled);
+ BOOST_CHECK(mt::funneled < mt::multiple);
+ BOOST_CHECK(mt::multiple > mt::funneled);
+
+ BOOST_CHECK(mt::serialized < mt::multiple);
+ BOOST_CHECK(mt::multiple > mt::serialized);
+
+ BOOST_CHECK(mt::single <= mt::single);
+ BOOST_CHECK(mt::single <= mt::funneled);
+ BOOST_CHECK(mt::funneled >= mt::single);
+ BOOST_CHECK(mt::single <= mt::serialized);
+ BOOST_CHECK(mt::serialized >= mt::single);
+ BOOST_CHECK(mt::single <= mt::multiple);
+ BOOST_CHECK(mt::multiple >= mt::single);
+
+ BOOST_CHECK(mt::funneled <= mt::funneled);
+ BOOST_CHECK(mt::funneled <= mt::serialized);
+ BOOST_CHECK(mt::serialized >= mt::funneled);
+ BOOST_CHECK(mt::funneled <= mt::multiple);
+ BOOST_CHECK(mt::multiple >= mt::funneled);
+
+ BOOST_CHECK(mt::serialized <= mt::serialized);
+ BOOST_CHECK(mt::serialized <= mt::multiple);
+ BOOST_CHECK(mt::multiple >= mt::serialized);
+
+ BOOST_CHECK(mt::multiple <= mt::multiple);
+}
+
+int
+test_main(int argc, char* argv[]) {
+ test_threading_levels_io();
+ test_threading_level_cmp();
+ return 0;
+}
Index: libs/mpi/test/Jamfile.v2
===================================================================
--- libs/mpi/test/Jamfile.v2 (révision 82588)
+++ libs/mpi/test/Jamfile.v2 (copie de travail)
@@ -23,6 +23,11 @@
[ mpi-test broadcast_test : : : 2 17 ]
[ mpi-test gather_test ]
[ mpi-test is_mpi_op_test : : : 1 ]
+ [ mpi-test mt_level_test : : : 1 ]
+ [ mpi-test mt_init_test-single : mt_init_test.cpp : <testing.arg>"single" : 1 4 ]
+ [ mpi-test mt_init_test-funneled : mt_init_test.cpp : <testing.arg>"funneled" : 1 4 ]
+ [ mpi-test mt_init_test-serialized : mt_init_test.cpp : <testing.arg>"serialized" : 1 4 ]
+ [ mpi-test mt_init_test-multiple : mt_init_test.cpp : <testing.arg>"multiple" : 1 4 ]
# Note: Microsoft MPI fails nonblocking_test on 1 processor
[ mpi-test nonblocking_test ]
[ mpi-test reduce_test ]
Index: libs/mpi/doc/mpi.qbk
===================================================================
--- libs/mpi/doc/mpi.qbk (révision 82588)
+++ libs/mpi/doc/mpi.qbk (copie de travail)
@@ -350,8 +350,6 @@
return 0;
}
-
-
[section:point_to_point Point-to-Point communication]
As a message passing library, MPI's primary purpose is to routine
@@ -1885,7 +1883,7 @@
`skeleton_proxy` objects can be received on the other end via `recv()`,
which stores a newly-created instance of your data structure with the
-same "shape" as the sender in its `"object` attribute:
+same "shape" as the sender in its `"object"` attribute:
shape = mpi.world.recv(0, 0)
my_data_structure = shape.object
@@ -1906,7 +1904,6 @@
The skeleton/content mechanism is a structured way to exploit the
interaction between custom-built MPI datatypes and `MPI_BOTTOM`, to
eliminate extra buffer copies.
-[endsect]
[section:python_compatbility C++/Python MPI Compatibility]
Boost.MPI is a C++ library whose facilities have been exposed to Python
@@ -1989,6 +1986,42 @@
and the C MPI library.
[endsect]
+[section:threading Threads]
+
+There are an increasing number of hybrid parrallel applications that mix
+distributed and shared memory parallelism. To know how to support that model,
+one need to know what level of threading support is guaranteed by the MPI
+implementation. There are 4 ordered level of possible threading support described
+by [classref boost::mpi::threading::level mpi::threading::level].
+At the lowest level, you should not use threads at all, at the highest level, any
+thread can perform MPI call.
+
+If you want to use multi-threading in your MPI application, you should indicate
+in the environment constructor your preffered threading support. Then probe the
+one the librarie did provide, and decide what you can do with it (it could be
+nothing, then aborting is a valid option):
+
+ #include <boost/mpi/environment.hpp>
+ #include <boost/mpi/communicator.hpp>
+ #include <iostream>
+ namespace mpi = boost::mpi;
+ namespace mt = mpi::threading;
+
+ int main()
+ {
+ mpi::environment env(mt::funneled);
+ if (env.thread_level() < mt::funneled) {
+ env.abort(-1);
+ }
+ mpi::communicator world;
+ std::cout << "I am process " << world.rank() << " of " << world.size()
+ << "." << std::endl;
+ return 0;
+ }
+
+
+[endsect]
+
[section:performance Performance Evaluation]
Message-passing performance is crucial in high-performance distributed
Index: libs/mpi/src/environment.cpp
===================================================================
--- libs/mpi/src/environment.cpp (révision 82588)
+++ libs/mpi/src/environment.cpp (copie de travail)
@@ -9,11 +9,58 @@
#include <boost/mpi/exception.hpp>
#include <boost/mpi/detail/mpi_datatype_cache.hpp>
#include <cassert>
+#include <string>
#include <exception>
#include <stdexcept>
+#include <ostream>
namespace boost { namespace mpi {
+namespace threading {
+std::istream& operator>>(std::istream& in, level& l)
+{
+ std::string tk;
+ in >> tk;
+ if (!in.bad()) {
+ if (tk == "single") {
+ l = single;
+ } else if (tk == "funneled") {
+ l = funneled;
+ } else if (tk == "serialized") {
+ l = serialized;
+ } else if (tk == "multiple") {
+ l = multiple;
+ } else {
+ in.setstate(std::ios::badbit);
+ }
+ }
+ return in;
+}
+std::ostream& operator<<(std::ostream& out, level l)
+{
+ switch(l) {
+ case single:
+ out << "single";
+ break;
+ case funneled:
+ out << "funneled";
+ break;
+ case serialized:
+ out << "serialized";
+ break;
+ case multiple:
+ out << "multiple";
+ break;
+ default:
+ out << "<level error>[" << int(l) << ']';
+ out.setstate(std::ios::badbit);
+ break;
+ }
+ return out;
+}
+
+} // namespace threading
+
#ifdef BOOST_MPI_HAS_NOARG_INITIALIZATION
environment::environment(bool abort_on_exception)
: i_initialized(false),
@@ -26,6 +73,21 @@
MPI_Errhandler_set(MPI_COMM_WORLD, MPI_ERRORS_RETURN);
}
+
+environment::environment(threading::level mt_level, bool abort_on_exception)
+ : i_initialized(false),
+ abort_on_exception(abort_on_exception)
+{
+ // It is not clear that we can pass null in MPI_Init_thread.
+ int dummy_thread_level = 0;
+ if (!initialized()) {
+ BOOST_MPI_CHECK_RESULT(MPI_Init_thread,
+ (0, 0, int(mt_level), &dummy_thread_level ));
+ i_initialized = true;
+ }
+
+ MPI_Errhandler_set(MPI_COMM_WORLD, MPI_ERRORS_RETURN);
+}
#endif
environment::environment(int& argc, char** &argv, bool abort_on_exception)
@@ -40,6 +102,22 @@
MPI_Errhandler_set(MPI_COMM_WORLD, MPI_ERRORS_RETURN);
}
+environment::environment(int& argc, char** &argv, threading::level mt_level,
+ bool abort_on_exception)
+ : i_initialized(false),
+ abort_on_exception(abort_on_exception)
+{
+ // It is not clear that we can pass null in MPI_Init_thread.
+ int dummy_thread_level = 0;
+ if (!initialized()) {
+ BOOST_MPI_CHECK_RESULT(MPI_Init_thread,
+ (&argc, &argv, int(mt_level), &dummy_thread_level));
+ i_initialized = true;
+ }
+
+ MPI_Errhandler_set(MPI_COMM_WORLD, MPI_ERRORS_RETURN);
+}
+
environment::~environment()
{
if (i_initialized) {
@@ -122,4 +200,20 @@
return std::string(name, len);
}
+threading::level environment::thread_level()
+{
+ int level;
+
+ BOOST_MPI_CHECK_RESULT(MPI_Query_thread, (&level));
+ return static_cast<threading::level>(level);
+}
+
+bool environment::main_thread()
+{
+ int isit;
+
+ BOOST_MPI_CHECK_RESULT(MPI_Is_thread_main, (&isit));
+ return bool(isit);
+}
+
} } // end namespace boost::mpi
Index: tools/build/v2/tools/mpi.jam
===================================================================
--- tools/build/v2/tools/mpi.jam (révision 82588)
+++ tools/build/v2/tools/mpi.jam (copie de travail)
@@ -314,7 +314,7 @@
# Prepend COMPILER as the executable name, to match the format of
# other compilation commands.
- compile_flags = "COMPILER $(compile_flags)" ;
+ compile_flags = "COMPILER $(compile_flags) -DOMPI_SKIP_MPICXX " ;
link_flags = "COMPILER $(link_flags)" ;
}
# Look for LAM-MPI's -showme
_______________________________________________
Boost-mpi mailing list
[email protected]
http://lists.boost.org/mailman/listinfo.cgi/boost-mpi
