Hi Open MPI developers, Though Sylvain's original mail (*1) was sent 4 months ago and nobody replied to it, I'm interested in this issue and strongly agree with Sylvain.
*1 http://www.open-mpi.org/community/lists/devel/2010/01/7275.php As explained by Sylvain, current Open MPI implementation always returns MPI_THREAD_SINGLE as provided thread level if neither --enable-mpi-threads nor --enable-progress-threads was specified at configure (v1.4). If we use OpenMP with MPI, we need at least MPI_THREAD_FUNNELED even if MPI functions are called only outside of omp parallel region, like below. #pragma omp parallel for for (...) { /* computation */ } MPI_Allreduce(...); #pragma omp parallel for for (...) { /* computation */ } This means Open MPI users must specify --enable-mpi-threads or --enable-progress-threads to use OpenMP. Is it true? But this two configure options, i.e. OMPI_HAVE_THREAD_SUPPORT macro, lead to performance penalty by mutex lock/unlock. # I know that we can actually use OpenMP with MPI_THREAD_SINGLE. # I'm talking about reason why Open MPI returns MPI_THREAD_SINGLE. I believe OMPI_HAVE_THREADS (not OMPI_HAVE_THREAD_SUPPORT !) is sufficient to support MPI_THREAD_FUNNELED and MPI_THREAD_SERIALIZED, and therefore OMPI_HAVE_THREAD_SUPPORT should be OMPI_HAVE_THREADS at following part in ompi_mpi_init function, as suggested by Sylvain. ompi_mpi_thread_requested = requested; if (OMPI_HAVE_THREAD_SUPPORT == 0) { /* <- should be OMPI_HAVE_THREADS ? */ ompi_mpi_thread_provided = *provided = MPI_THREAD_SINGLE; ompi_mpi_main_thread = NULL; } else if (OMPI_ENABLE_MPI_THREADS == 1) { ompi_mpi_thread_provided = *provided = requested; ompi_mpi_main_thread = opal_thread_get_self(); } else { if (MPI_THREAD_MULTIPLE == requested) { ompi_mpi_thread_provided = *provided = MPI_THREAD_SERIALIZED; } else { ompi_mpi_thread_provided = *provided = requested; } ompi_mpi_main_thread = opal_thread_get_self(); } If not, what is the problem to support MPI_THREAD_FUNNELED and MPI_THREAD_SERIALIZED without OMPI_HAVE_THREAD_SUPPORT? Though options and code above are those of v1.4, trunk seems to same. MPI_THREAD_MULTIPLE needs --enable-opal-multi-threads or --enable-mpi-thread-multiple and they lead to performance penalty. Regards, Kawashima > Hi list, > > I'm currently playing with thread levels in Open MPI and I'm quite > surprised by the current code. > > First, the C interface : > at ompi/mpi/c/init_thread.c:56 we have : > #if OPAL_ENABLE_MPI_THREADS > *provided = MPI_THREAD_MULTIPLE; > #else > *provided = MPI_THREAD_SINGLE; > #endif > prior to the call to ompi_mpi_init() which will in turn override the > "provided" value. Should we remove these 5 lines ? > > Then at ompi/runtime/ompi_mpi_init.c:372, we have -I guess- the real code > which is : > > ompi_mpi_thread_requested = requested; > if (OPAL_HAVE_THREAD_SUPPORT == 0) { > ompi_mpi_thread_provided = *provided = MPI_THREAD_SINGLE; > ompi_mpi_main_thread = NULL; > } else if (OPAL_ENABLE_MPI_THREADS == 1) { > ompi_mpi_thread_provided = *provided = requested; > ompi_mpi_main_thread = opal_thread_get_self(); > } else { > if (MPI_THREAD_MULTIPLE == requested) { > ompi_mpi_thread_provided = *provided = MPI_THREAD_SERIALIZED; > } else { > ompi_mpi_thread_provided = *provided = requested; > } > ompi_mpi_main_thread = opal_thread_get_self(); > } > > This code seems ok to me provided that : > * (OPAL_ENABLE_MPI_THREADS == 1) means "Open MPI configured to provide > thread multiple", > * (OPAL_HAVE_THREAD_SUPPORT == 0) means "we do not have threads at all" > though even if we do not have threads at compile time, it does in no way > prevent us from doing THREAD_FUNNELED or THREAD_SERIALIZED. > > The reality seems different at opal/include/opal_config_bottom.h:70 : > > /* Do we have posix or solaris thread lib */ > #define OPAL_HAVE_THREADS (OPAL_HAVE_POSIX_THREADS || > OPAL_HAVE_SOLARIS_THREADS) > /* Do we have thread support? */ > #define OPAL_HAVE_THREAD_SUPPORT (OPAL_ENABLE_MPI_THREADS || > OPAL_ENABLE_PROGRESS_THREADS) > > "we do not have threads at all" seems to me to be OPAL_HAVE_THREADS and > not OPAL_HAVE_THREAD_SUPPORT. What do you think ? Maybe > OPAL_HAVE_THREAD_SUPPORT should be renamed, too (seems misleading to me). > > The result is that the current default configuration of Open MPI has > OPAL_HAVE_THREAD_SUPPORT defined to 0 and Open MPI always returns > THREAD_SINGLE, even if it is perfectly capable of THREAD_FUNNELED and > THREAD_SERIALIZED. > > Sylvain
