Re: [OMPI users] NUMA interaction with Open MPI

be ignored by processes placed in that cgroup. The latter is a user-changeable binding that must be inside the former. Brice Le 19/07/2017 17:29, Iliev, Hristo a écrit : > Giles, > > Mems_allowed_list has never worked for me: > > $ uname -r > 3.10.0-514.26.1.e17.x86_64 >

Re: [OMPI users] NUMA interaction with Open MPI

Giles, Mems_allowed_list has never worked for me: $ uname -r 3.10.0-514.26.1.e17.x86_64 $ numactl -H | grep available available: 2 nodes (0-1) $ grep Mems_allowed_list /proc/self/status Mems_allowed_list: 0-1 $ numactl -m 0 grep Mems_allowed_list /proc/self/status Mems_allowed_list:

Re: [OMPI users] Severe performance issue with PSM2 and single-node CP2K jobs

a similar problem here and I started a thread a few days ago. https://mail-archive.com/users@lists.open-mpi.org/msg30581.html Regard, Jingchao _ From: users <users-boun...@lists.open-mpi.org> on behalf of Iliev, Hristo <il...@itc.rwth-aachen.de> Sent: Wednesday, February 8, 2017

[OMPI users] Severe performance issue with PSM2 and single-node CP2K jobs

Hi, While trying to debug a severe performance regression of CP2K runs with Open MPI 1.10.4 on our new cluster, after reproducing the problem with single-node jobs too, we found out that the root cause is that the presence of Intel Omni-Path hardware triggers the use of the cm PML and

Re: [OMPI users] open-mpi on Mac OS 10.9 (Mavericks)

Karl, The Xcode command-line tools provide a g++ command that is effectively clang++. Since mpic++ invokes g++ without the full path to it, it might pick the wrong g++. Make sure that /opt/local/bin is the first item in your $PATH. Hristo -- Hristo Iliev, PhD - High Performance Computing Team /

Re: [OMPI users] debugging performance regressions between versions

Hi Dave, Is it MPI_ALLTOALL or MPI_ALLTOALLV that runs slower? If it is the latter, the reason could be that the default implementation of MPI_ALLTOALLV in 1.6.5 is different from that in 1.5.4. To switch back to the previous one, use: --mca coll_tuned_use_dynamic_rules 1 --mca

Re: [OMPI users] (no subject)

Hi, When all processes run on the same node they communicate via shared memory which delivers both high bandwidth and low latency. InfiniBand is slower and more latent than shared memory. Your parallel algorithm might simply be very latency sensitive and you should profile it with something like

Re: [OMPI users] knem/openmpi performance?

> -Original Message- > From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] > On Behalf Of Dave Love > Sent: Thursday, July 18, 2013 1:22 PM > To: Open MPI Users > Subject: Re: [OMPI users] knem/openmpi performance? > > Paul Kapinos writes: > >

Re: [OMPI users] Sandy Bridge performance question

VampirTrace works great together with PAPI in order to collect performance counter readings and to combine them with the function call/MPI traces. Then Vampir's "Performance Radar" feature can be used to quickly spot bottlenecks in your application. The only downside is that this kind of

Re: [OMPI users] Possible bug in OpenMPI 1.6.3 in selecting v4 or v6 network addresses on the same adaptor

Hi, You can use the btl_tcp_disable_family and oob_tcp_disable_family MCA parameters to disable the use of a specific IP family addresses. Set both parameters to 6 to disable IPv6 or set them both to 4 in order to disable IPv4. Kind regards, Hristo > -Original Message- > From:

Re: [OMPI users] bug in mpif90? OMPI_FC envvar does not work with 'use mpi'

n bug" in that particular FAQ entry. Dominik On 03/13/2013 09:58 AM, Iliev, Hristo wrote: Hi, Dominik, gfortran and ifort produce Fortran module files (*.mod) with completely different and hence incompatible formats. You cannot use a module compiled with gfortran in ifort and vice versa. That'

Re: [OMPI users] bug in mpif90? OMPI_FC envvar does not work with 'use mpi'

Hi, Dominik, gfortran and ifort produce Fortran module files (*.mod) with completely different and hence incompatible formats. You cannot use a module compiled with gfortran in ifort and vice versa. That's why many Fortran libraries come with explicit module interface definitions that have to be

Re: [OMPI users] All_to_allv algorithm patch

Hi, This is the users mailing list. There is a separate one for questions related to Open MPI development - de...@open-mpi.org. Besides, why don't you open a ticket in the Open MPI Trac at https://svn.open-mpi.org/trac/ompi/ and post there patches against trunk? My experience shows that even

Re: [OMPI users] Initializing OMPI with invoking the array constructor on Fortran derived types causes the executable to crash

Some MPI libraries (including OMPI and IMPI) hook the system memory management routines like 'malloc' and 'free' (used behind the scenes by Fortran on Unix). This is usually performed in order to manage memory registration for RDMA-based networks like InfiniBand. I would guess that Open MPI

Re: [OMPI users] Initializing OMPI with invoking the array constructor on Fortran derived types causes the executable to crash

Hi, Gus, Automatic allocation (an reallocation) on assignment is among the nifty features of Fortran 2003. In this case "conc" is automatically allocated so to match the shape of its initialiser array "[ xx, yy ]". Note that "xx" and "yy" are not allocatable though their derived type has an

Re: [OMPI users] MPI_Alltoallv performance regression 1.6.0 to 1.6.1

h pairwise exchange. This is > >> O(comm_size) > >> waits, may of which are zero. I'm not clear what optimizations there > >> are for zero-size isend/irecv, but surely there's a great deal more > >> latency if each pairwise exchange has to be confirmed complet

Re: [OMPI users] 0xc000007b error exit on 64-bit Windows 7

Hi, 0xC07B is STATUS_INVALID_IMAGE_FORMAT. It mostly means that some of the dynamic link libraries (DLLs) that the executable is linked against are of different "bitness", e.g. 32-bit. It could be a packaging error in Open MPI, or it could be messed up installation. You could use the

Re: [OMPI users] MPI_Alltoallv performance regression 1.6.0 to 1.6.1

Hi, Simon, The pairwise algorithm passes messages in a synchronised ring-like fashion with increasing stride, so it works best when independent communication paths could be established between several ports of the network switch/router. Some 1 Gbps Ethernet equipment is not capable of doing so,

Re: [OMPI users] Setting RPATH for Open MPI libraries

Hi, Reuti, Here is an informative article on dynamic libraries path in OS X: https://blogs.oracle.com/dipol/entry/dynamic_libraries_rpath_and_mac The first comment is very informative too. Kind regards, Hristo -- Hristo Iliev, Ph.D. -- High Performance Computing RWTH Aachen University, Center

Re: [OMPI users] Problems with btl openib and MPI_THREAD_MULTIPLE

Hello, Markus, The openib BTL component is not thread-safe. It disables itself when the thread support level is MPI_THREAD_MULTIPLE. See this rant from one of my colleagues: http://www.open-mpi.org/community/lists/devel/2012/10/11584.php A message is shown but only if the library was compiled

Re: [OMPI users] openmpi-1.6.2 and registerable memory

Hi, This question was answered by Yevgeny Kliteynik from Mellanox on the developers list. The amount of registerable memory should be about twice the size of the physical memory because of the way physical memory is being registered with InfiniBand HCAs, not because of possible overcommitment.

Re: [OMPI users] unacceptable latency in gathering process

Hi, I would suggest that (if you haven't done it already) you trace your program's execution with Vampir or Scalasca. The latter has some pretty nice analysis capabilities built-in and can detect common patterns that would make your code not to scale, no matter how good the MPI library is. Also

Re: [OMPI users] A question on MPI_Probe

Hi, I believe Jeff Squyres has already answered your question on the Open MPI mailing list, or at least hinted on the possible problem. MPI messages are received in the order they were sent but only within the specific (tag, communicator) tuple. This basically means: - within the same

Re: [OMPI users] About MPI_TAG_UB

en MPI might break it. Just my €0.02. Kind regards, Hristo Iliev -- Hristo Iliev, Ph.D. -- High Performance Computing RWTH Aachen University, Center for Computing and Communication Rechen- und Kommunikationszentrum der RWTH Aachen Seffenter Weg 23, D 52074 Aachen (Germany) > -O

Re: [OMPI users] About MPI_TAG_UB

the routing info inside the message payload. Best regards, Hristo Iliev -- Hristo Iliev, Ph.D. -- High Performance Computing RWTH Aachen University, Center for Computing and Communication Rechen- und Kommunikationszentrum der RWTH Aachen Seffenter Weg 23, D 52074 Aachen (Germany) > -Original Mess

Re: [OMPI users] Algorithms used in MPI_BCast

Hello, Are you making these rules manually or in some automated way? Regards, Pavel Mezentsev. 2012/9/21 Iliev, Hristo <il...@rz.rwth-aachen.de> Hi, Open MPI uses several different algorithms depending on which module from the "coll" framework is selected to p

Re: [OMPI users] Algorithms used in MPI_BCast

Hi, Open MPI uses several different algorithms depending on which module from the "coll" framework is selected to perform the logic of the broadcast operation. The "tuned" module which usually gets selected on cluster installations uses a heuristic rule, based on the message size and the size of

Re: [OMPI users] Measuring latency

Hello, Intel MPI Benchmarks suite (http://software.intel.com/en-us/articles/intel-mpi-benchmarks/) will probably measure more things about your MPI environment than you'd ever need to know :) NetPIPE (http://www.scl.ameslab.gov/netpipe/) also has an MPI version. It measures point-to-point

Re: [OMPI users] MPI_Irecv: Confusion with <> inputy parameter

have decided to receive a > large payload in chunks of 80 bytes? > > I am sorry, the manual and the API reference was too vague for me. > > Thanks a lot > > Devendra > From: "Iliev, Hristo" <il...@rz.rwth-aachen.de> > To: Open MPI Users <us...@

Re: [OMPI users] MPI_Irecv: Confusion with <> inputy parameter

Jeff, >> Or is it the number of elements that are expected to be received, and hence >> MPI_Test will tell me that the receive is not complete untill "count" number >> of elements have not been received? > > Yes. Answering "Yes" this question might further the confusion there. The "count"

Re: [OMPI users] issue with addresses

the list at > users-ow...@open-mpi.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of users digest..." > > > Today's Topics: > >1. Efficient polling for both incoming messages and request &g

Re: [OMPI users] issue with addresses

Hello, Placement of data in memory is highly implementation dependent. I assume you are running on Linux. This OS’ libc (glibc) provides two different methods for dynamic allocation of memory – heap allocation and anonymous mappings. Heap allocation is used for small data up to MMAP_TRESHOLD

Re: [OMPI users] mpirun command gives ERROR

Hi, You should consult the CPMD manual on how to run the program in parallel - this doesn't look like a problem in Open MPI. The error comes from MPI_ABORT being called by rank 0. As rank 0 process is the one that reads all the input data and prepares the computation I would say that the most

[OMPI users] Fwd: MPI Problem

t; -Wl,-rpath,/opt/MPI/openmpi-1.5.3/linux/gcc/lib >> -I/opt/MPI/openmpi-1.5.3/linux/gcc/lib >> -L/opt/MPI/openmpi-1.5.3/linux/gcc/lib -lmpi_f90 -lmpi_f77 -lmpi -ldl >> -Wl,--export-dynamic -lnsl -lutil -lm -ldl -x f95-cpp-input >> simul/simul_wrapper.f90 -o simul/simul_

Re: [OMPI users] MPI Problem

I will appreciate if he would respond to my earlier inquiry as he appears to be a user of our computing facilities. -- Hristo Iliev, Ph.D. -- High Performance Computing RWTH Aachen University, Center for Computing and Communication > -Original Message- > From: users-boun...@open-mpi.org

Re: [OMPI users] MPI Problem

Hi, Could you please compare the output from "mpif77/mpif90 -showme" when executed from the command line and from inside the Makefile? Could you also check that the argument to MPI_INIT is a normal INTEGER and is not getting some funny KIND or preprocessor treatment? Regards, -- Hristo

Re: [OMPI users] two jobs in a single mpirun command and MPI_COMM_WORLD issue ...

Hi, I think you misunderstood what a MIMD launch with mpirun/mpiexec actually does. mpirun -np X prog1 : -np Y prog2 starts a *single* MPI job consisting of X+Y processes in total of which the X processes execute prog1 and Y processes execute prog2 but they still belong to the same MPI job and