Re: [OMPI users] OpenMPI Segfault when psm is enabled?

2017-03-15 Thread Saliya Ekanayake
Thank you, Gilles. Saliya On Sun, Mar 12, 2017 at 3:36 AM, Gilles Gouaillardet < gilles.gouaillar...@gmail.com> wrote: > PSM is the infinipath driver, so unless you have some infinipath hardware, > you can safely disable it > > Cheers, > > Gilles > > > On Sunday, March 12, 2017, Saliya Ekanayake

Re: [OMPI users] openib/mpi_alloc_mem pathology

2017-03-15 Thread Jeff Hammond
On Wed, Mar 15, 2017 at 5:44 PM Jeff Squyres (jsquyres) wrote: > On Mar 15, 2017, at 8:25 PM, Jeff Hammond wrote: > > > > I couldn't find the docs on mpool_hints, but shouldn't there be a way to > disable registration via MPI_Info rather than patching the source? > > Yes; that's what I was think

Re: [OMPI users] openib/mpi_alloc_mem pathology

2017-03-15 Thread Jeff Squyres (jsquyres)
On Mar 15, 2017, at 8:25 PM, Jeff Hammond wrote: > > I couldn't find the docs on mpool_hints, but shouldn't there be a way to > disable registration via MPI_Info rather than patching the source? Yes; that's what I was thinking, but wanted to get the data point first. Specifically: if this tes

Re: [OMPI users] openib/mpi_alloc_mem pathology

2017-03-15 Thread Jeff Hammond
I couldn't find the docs on mpool_hints, but shouldn't there be a way to disable registration via MPI_Info rather than patching the source? Jeff PS Jeff Squyres: ;-) ;-) ;-) On Wed, Mar 15, 2017 at 5:03 PM, Jeff Squyres (jsquyres) wrote: > > It looks like there were 3 separate threads on this C

Re: [OMPI users] openib/mpi_alloc_mem pathology

2017-03-15 Thread Jeff Squyres (jsquyres)
It looks like there were 3 separate threads on this CP2K issue, but I think we developers got sidetracked because there was a bunch of talk in the other threads about PSM, non-IB(verbs) networks, etc. So: the real issue is an app is experiencing a lot of slowdown when calling MPI_ALLOC_MEM/MPI_

Re: [OMPI users] openib/mpi_alloc_mem pathology

2017-03-15 Thread Dave Love
Paul Kapinos writes: > Nathan, > unfortunately '--mca memory_linux_disable 1' does not help on this > issue - it does not change the behaviour at all. > Note that the pathological behaviour is present in Open MPI 2.0.2 as > well as in /1.10.x, and Intel OmniPath (OPA) network-capable nodes are >

Re: [OMPI users] Mellanox EDR performance

2017-03-15 Thread gilles
Thanks for sharing your findings. just to be clear, your application is running at full speed. only MPI_Wtime() is busted, so timers used internally in your app might mislead you and suggests performance are worst than what they really are Cheers, Gilles - Original Message - I'm

Re: [OMPI users] Mellanox EDR performance

2017-03-15 Thread Yong Qin
I'm not sure why nobody has encountered this issue on the mailing list. After some fiddling I was finally able to isolated to a performance regression introduced between 2.0.1 and 2.0.2. While trying to binary search the exact commit causing the performance regression my colleague brought this to m