Different MPI implementations have different capabilities. Some can handle multiple threads making MPI calls in parallel (MPI_THREAD_MULTIPLE), some require only one thread accessing MPI at a time (MPI_THREAD_SERIALIZED), and some only work if all calls to MPI go through the same thread (MPI_THREAD_FUNNELED: other threads can exist, but should never make MPI calls), or if there is only one thread in the program (MPI_THREAD_SINGLE).
Our PAMI backend can support the FUNNELED case, by using a special thread which handles all probing, and other threads can hand off message sending operations to that same thread. But we don't currently support this same capability in the MPI backend, since it's such a rare case and more difficult to implement. We do support the other three modes. See https://www.open-mpi.org/doc/v2.0/man3/MPI_Init_thread.3.php for more details on the different modes. Since your system only supports MPI_THREAD_SINGLE, you can only have one thread per place, which as you noted earlier, requires the combination of X10_STATIC_THREADS=1 X10_NTHREADS=1 and X10_NUM_IMMEDIATE_THREADS=0. Unfortunately, as you also noted, this means you can't use clocks in your program. I believe the only path forward here is to re-work your application so that it doesn't use clocks. - Ben From: Marina Andric <marina.and...@imtlucca.it> To: Mailing list for users of the X10 programming language <x10-users@lists.sourceforge.net> Date: 07/27/2016 03:05 PM Subject: Re: [X10-users] Problem with compiling programs to C++ backend after building x10 from source Hi Josh, I checked and the level provided is not serialized, as I thought (sorry), it is MPI_THREAD_SINGLE. Is it compatible with X10 implementation? Thanks a lot, Marina On Wed, Jul 27, 2016 at 7:20 PM, Joshua J Milthorpe <jjmil...@us.ibm.com> wrote: Hi Marina, it looks like you are setting the environment variable correctly. I guess it's possible that your MPI implementation doesn't support MPI_THREAD_SERIALIZED, either. What version of MPI are you using? (and what sort of cluster are you running on?) Can you try a simple MPI program that initializes MPI with MPI_THREAD_SERIALIZED, to see if this works outside of X10? https://www.mpi-forum.org/docs/mpi-3.1/mpi31-report/node303.htm Cheers, Josh Inactive hide details for Marina Andric ---07/27/2016 12:53:35 PM---Hi Josh, Thanks a lot for your email.Marina Andric ---07/27/2016 12:53:35 PM---Hi Josh, Thanks a lot for your email. From: Marina Andric <marina.and...@imtlucca.it> To: Mailing list for users of the X10 programming language < x10-users@lists.sourceforge.net> Date: 07/27/2016 12:53 PM Subject: Re: [X10-users] Problem with compiling programs to C++ backend after building x10 from source Hi Josh, Thanks a lot for your email. I just tested it again and faced the old mistake again: "Underlying MPI implementation does not provide requested threading level Unable to support requested level of X10 threading; exiting" I am struggling to understand why this happens. I opened x10rt_mpi.cc file and saw the part of the code that generates the error, which is when level_required != level_provided, where level_required is set according to what getenv(X10RT_MPI_THREAD_SERIALIZED) returns. Am I maybe setting the variable in a wrong way? I am executing a script with a line export X10RT_MPI_THREAD_SERIALIZED=1 Thanks, Marina On Wed, Jul 27, 2016 at 6:29 PM, Joshua J Milthorpe <jjmil...@us.ibm.com> wrote: Hi Marina, the message you found regarding X10RT_MPI_THREAD_MULTIPLE is now out of date :-( Since X10 2.5.2, MPI_THREAD_MULTIPLE is the default threading level, but it is possible to request MPI_THREAD_SERIALIZED by setting the environment variable X10RT_MPI_THREAD_SERIALIZED. In this case, X10 will serialize all calls to MPI functions through a global lock (in accordance with the rules for MPI_THREAD_SERIALIZED). https://xtenlang.atlassian.net/browse/XTENLANG-3490 Could you please try running with X10RT_MPI_THREAD_SERIALIZED=1 and let us know the result? Cheers, Josh Milthorpe Post Doctoral Researcher Cognitive Systems: Learning to Reason IBM Research Phone: 1-914-945-2209 1101 Kitchawan Rd E-mail: jjmil...@us.ibm.com Yorktown Heights, NY 10598 United States Inactive hide details for Marina Andric ---07/27/2016 12:13:16 PM---Hi again, I came acrossMarina Andric ---07/27/2016 12:13:16 PM---Hi again, I came across From: Marina Andric <marina.and...@imtlucca.it> To: Mailing list for users of the X10 programming language < x10-users@lists.sourceforge.net> Date: 07/27/2016 12:13 PM Subject: Re: [X10-users] Problem with compiling programs to C++ backend after building x10 from source Hi again, I came across "Only if X10_STATIC_THREADS=1 X10_NTHREADS=1 and X10_NUM_IMMEDIATE_THREADS=0 will we use single-threaded MPI modes." on one of the pages dedicated to X10. I now initialized the variables and I'm not anymore facing the error that provided and requested threading levels do not match. However, my code uses Clocks for synchronization and I get runtime error ClockUseException: Clocks are not compatible with static threads. I guess that I've hit the wall now and can't proceed in this direction. Or is there a possible workaround to make Clocks work in mpi thread serialized setting ? Thank you, Marina On Wed, Jul 27, 2016 at 3:26 PM, Marina Andric < marina.and...@imtlucca.it> wrote: Hi, Thanks a lot Marco for the help. Yes, I initially copied the executable, then I built it on the cluster and now it works, almost :) Works well until the moment I compile the program with -x10rt mpi option. The program is compiled and executable is generated, but it is not running. The error states: "Underlying MPI implementation does not provide requested threading level Unable to support requested level of X10 threading; exiting" (the whole output is attached) I found this page: https://sourceforge.net/p/x10/mailman/message/33490796/ and started looking into X10 runtime variables (X10RT_MPI_THREAD_MULTIPLE, X10_STATIC_THREADS) and played a bit with those but with no success.. On my linux laptop it works with -x10rt mpi option, with no errors. In both cases, on my machine and the cluster, MPI_THREAD_MULTIPLE is set to no. Does someone has an idea how to solve this? Thank you, Marina On Tue, Jul 26, 2016 at 1:00 AM, Marco Bungart < m.bung...@gmx.net> wrote: Hi Maina, looks like you have not built the program on the cluster. Did you just copy the executable? This will in general not work. You should download and compile/install X10 on the cluster as well. The easiest way would be to build X10 from source (tutorial found here: http://x10-lang.org/x10-development/building-x10-from-source.html ). After that, re-compile your program on the cluster and execute it. Cheers, Marco Am 25.07.2016 um 18:11 schrieb Marina Andric: > > Thanks a lot Marco, this was something I neglected :) the program is > compiling/executing now on my machine. > > 'Im still having a problem on the cluster. I get compilation error stating: > > x10.dist/stdlib/lib/libx10.so: undefined reference to 'memcpy@GLIBC_2.14' > x10.dist/stdlib/lib/libx10.so: undefined reference to > 'clock_gettime@GLIBC_2.17' > > Does somebody know the possible origins of this problem? Do I maybe need > to load some additional modules on the cluster, except gcc and mpi? > > Warm regards, > Marina > > > > > > > > > > > > On Sun, Jul 24, 2016 at 11:48 PM, Marco Bungart < m.bung...@gmx.net > <mailto:m.bung...@gmx.net>> wrote: > > Hi, > > have you tried running "$> ./a.out" instead of "$> a.out"? Some systems > distinguish between system-wide executable (i.e. executable located in > one of the paths specified in the $PATH environment variable) and > executables in the current folder (i.e. "."). > > Cheers, > Marco > > Am 24.07.2016 um 21:22 schrieb Marina Andric: > > Hi all, > > > > I am trying to run an x10 program on a cluster. The cluster supports > > only MPI based communication between nodes so I built the latest x10 > > source (version 2.6.0) with option -DX10RT_MPI=true and the build was > > successful. > > > > My program works when it's compiled to java (via x10c) but it doesn't > > work when it is compiled to c++ (via x10c++). > > > > For example, when > > > >> x10.dist/bin/x10c++ hello.x10 (creates a.out) > >> a.out > > > > the outcome is a.out: command not found. > > > > I tested a bit around, with -o option, -x10rt sockets (mpi) and > nothing > > seemed to change. I eventually built x10 without -DX10RT_MPI=true and > > the problem is the same, also with the x10 version 2.5.4. > > > > I was building and testing on my local linux machine and I checked all > > the program versions and they seem to be ok: > > > > ant 1.9.3 > > g++ 4.8.4 > > > > Can someone please say if I'm missing something? > > > > Thank you, > > Marina > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------------ > > What NetFlow Analyzer can do for you? Monitors network bandwidth > and traffic > > patterns at an interface-level. Reveals which users, apps, and > protocols are > > consuming the most bandwidth. Provides multi-vendor support for > NetFlow, > > J-Flow, sFlow and other flows. Make informed decisions using > capacity planning > > reports.http://sdm.link/zohodev2dev > > > > > > > > _______________________________________________ > > X10-users mailing list > > X10-users@lists.sourceforge.net > <mailto:X10-users@lists.sourceforge.net> > > https://lists.sourceforge.net/lists/listinfo/x10-users > > > > --- > Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft. > https://www.avast.com/antivirus > > > ------------------------------------------------------------------------------ > What NetFlow Analyzer can do for you? Monitors network bandwidth and > traffic > patterns at an interface-level. Reveals which users, apps, and > protocols are > consuming the most bandwidth. Provides multi-vendor support for NetFlow, > J-Flow, sFlow and other flows. Make informed decisions using > capacity planning > reports.http://sdm.link/zohodev2dev > _______________________________________________ > X10-users mailing list > X10-users@lists.sourceforge.net <mailto: X10-users@lists.sourceforge.net> > https://lists.sourceforge.net/lists/listinfo/x10-users > > > > > ------------------------------------------------------------------------------ > What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic > patterns at an interface-level. Reveals which users, apps, and protocols are > consuming the most bandwidth. Provides multi-vendor support for NetFlow, > J-Flow, sFlow and other flows. Make informed decisions using capacity planning > reports.http://sdm.link/zohodev2dev > > > > _______________________________________________ > X10-users mailing list > X10-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/x10-users > --- Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft. https://www.avast.com/antivirus ------------------------------------------------------------------------------ What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic patterns at an interface-level. Reveals which users, apps, and protocols are consuming the most bandwidth. Provides multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make informed decisions using capacity planning reports.http://sdm.link/zohodev2dev _______________________________________________ X10-users mailing list X10-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/x10-users ------------------------------------------------------------------------------ What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic patterns at an interface-level. Reveals which users, apps, and protocols are consuming the most bandwidth. Provides multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make informed decisions using capacity planning reports. http://sdm.link/zohodev2dev_______________________________________________ X10-users mailing list X10-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/x10-users ------------------------------------------------------------------------------ What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic patterns at an interface-level. Reveals which users, apps, and protocols are consuming the most bandwidth. Provides multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make informed decisions using capacity planning reports.http://sdm.link/zohodev2dev _______________________________________________ X10-users mailing list X10-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/x10-users ------------------------------------------------------------------------------ What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic patterns at an interface-level. Reveals which users, apps, and protocols are consuming the most bandwidth. Provides multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make informed decisions using capacity planning reports. http://sdm.link/zohodev2dev_______________________________________________ X10-users mailing list X10-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/x10-users ------------------------------------------------------------------------------ What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic patterns at an interface-level. Reveals which users, apps, and protocols are consuming the most bandwidth. Provides multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make informed decisions using capacity planning reports.http://sdm.link/zohodev2dev _______________________________________________ X10-users mailing list X10-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/x10-users ------------------------------------------------------------------------------ _______________________________________________ X10-users mailing list X10-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/x10-users
------------------------------------------------------------------------------
_______________________________________________ X10-users mailing list X10-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/x10-users
