Re: [OMPI users] hwloc, OpenMPI and unsupported OSes and toolchains

[Ralph H Castain]

I don’t see how Open MPI can operate without pthreads

> On Mar 19, 2018, at 3:23 PM, Gregory (tim) Kelly  wrote:
> 
> Hello Everyone,
> I'm inquiring to find someone that can answer some multi-part questions about 
> hwloc, OpenMPI and an alternative OS and toolchain.  I have a project as part 
> of my PhD work, and it's not a simple, one-part question.  For brevity, I am 
> omitting details about the OS and toolchain, other than that neither are 
> supported.  If forced to choose between OpenMPI and the OS/toolchain, I am 
> likely to choose the OS/toolchain and pursue other avenues for 
> parallelization.  That's part of what I am trying to determine with my 
> inquiry.
> 
> To summarize some of the question areas:
> 
> 1) The OS I am working with does not support MP
> 2) nor does it support pthreads
> 3) the hardware is quad-core SoC with an integrated memory controller
> 4) I'd like to see if it possible to utilize hwloc and shmem to build an 
> asymmetric multi-processing system where only one core has I/O but the other 
> three can run the executable
> 
> This is a fairly dedicated system to be used for analyzing ODEs (disease 
> models).  The hardware is cheap ($200) and uses very little power (can run 
> off a 12v battery), and the toolchain and OS are all BSD-licensed (and 
> everything will be published under that license).
> 
> If someone is available for off-line discussion (to minimize unnecessary 
> traffic to the list), I'd be more than willing to summarize the conversation 
> and contribute it to the online documentation.
> 
> Thank you,
> tim
> -- 
> 
> "Nuclear power is a hell of a way to boil water."  -- Albert Einstein
> ___
> users mailing list
> users@lists.open-mpi.org
> https://lists.open-mpi.org/mailman/listinfo/users

___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

[OMPI users] hwloc, OpenMPI and unsupported OSes and toolchains

[Gregory (tim) Kelly]

Hello Everyone,
I'm inquiring to find someone that can answer some multi-part questions
about hwloc, OpenMPI and an alternative OS and toolchain.  I have a
project as part of my PhD work, and it's not a simple, one-part
question.  For brevity, I am omitting details about the OS and
toolchain, other than that neither are supported.  If forced to choose
between OpenMPI and the OS/toolchain, I am likely to choose the
OS/toolchain and pursue other avenues for parallelization.  That's part
of what I am trying to determine with my inquiry, and not a reflection 
on OpenMPI.


To summarize some of the question areas:

1) The OS I am working with does not support MP
2) nor does it support pthreads
3) the hardware is quad-core x86 SoC with an integrated memory controller
4) I'd like to see if it possible to utilize hwloc and shmem to build an
asymmetric multi-processing system where only one core has I/O but the
other three can run the executable

This is a fairly dedicated system to be used for analyzing ODEs (disease
models).  The hardware is cheap ($200) and uses very little power (can
run off a 12v battery), and the toolchain and OS are all BSD-licensed
(and everything will be published under that license).

If someone is available for off-line discussion (to minimize unnecessary
traffic to the list), I'd be more than willing to summarize the
conversation and contribute it to the online documentation.

Thank you,
tim
--

"Nuclear power is a hell of a way to boil water."  -- Albert Einstein

___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

[OMPI users] hwloc, OpenMPI and unsupported OSes and toolchains

[Gregory (tim) Kelly]

Hello Everyone,
I'm inquiring to find someone that can answer some multi-part questions 
about hwloc, OpenMPI and an alternative OS and toolchain.  I have a 
project as part of my PhD work, and it's not a simple, one-part 
question.  For brevity, I am omitting details about the OS and 
toolchain, other than that neither are supported.  If forced to choose 
between OpenMPI and the OS/toolchain, I am likely to choose the 
OS/toolchain and pursue other avenues for parallelization.  That's part 
of what I am trying to determine with my inquiry.


To summarize some of the question areas:

1) The OS I am working with does not support MP
2) nor does it support pthreads
3) the hardware is quad-core SoC with an integrated memory controller
4) I'd like to see if it possible to utilize hwloc and shmem to build an 
asymmetric multi-processing system where only one core has I/O but the 
other three can run the executable


This is a fairly dedicated system to be used for analyzing ODEs (disease 
models).  The hardware is cheap ($200) and uses very little power (can 
run off a 12v battery), and the toolchain and OS are all BSD-licensed 
(and everything will be published under that license).


If someone is available for off-line discussion (to minimize unnecessary 
traffic to the list), I'd be more than willing to summarize the 
conversation and contribute it to the online documentation.


Thank you,
tim
--

"Nuclear power is a hell of a way to boil water."  -- Albert Einstein
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

Re: [OMPI users] OpenMPI slow with Infiniband

[Gilles Gouaillardet]

Supun,


did you configure Open MPI with --disable-dlopen ?

It was previously reported that this option disable the patcher (memory 
registration),


which impacts performance negatively.


If yes, then I suggest you reconfigure (and rebuild) without this option 
and see if it helps



Cheers,


Gilles


On 3/21/2018 2:46 AM, Supun Kamburugamuve wrote:

Hi,

I'm trying to run a small benchmark with Infiniband and Ethernet to 
see the difference. I get strange results where OpenMPI seems to be 
slower with Infiniband than Ethernet. I'm using the 3.0.0 version.


Using the following parameters to enable Ethernet.

--mca btl ^openib --mca btl_tcp_if_include 172.29.200.0/22 



The slowdown is significant and I cannot explain why. Infiniband seems 
to be working fine. I ran some benchmarks on IB and it performs as 
expected.


Thanks,
Supun..



___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users


___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

Re: [OMPI users] Old version openmpi 1.2 support infiniband?

[Kaiming Ouyang]

Hi Jeff,
Thank you for your advice. I will contact the author for some suggestions.
I also notice I may port this old version library to new openmpi 3.0. I
will work on this soon. Thank you.

Kaiming Ouyang, Research Assistant.
Department of Computer Science and Engineering
University of California, Riverside
900 University Avenue, Riverside, CA 92521


On Wed, Mar 21, 2018 at 5:24 AM, Jeff Squyres (jsquyres)  wrote:

> You might want to take that library author's advice from their README:
>
> -
> The source code herein was used as the basis of Rountree ICS 2009.  It
> was my first nontrivial MPI tool and was never intended to be released
> to the wider world.  I beleive it was tied rather tightly to a subset
> of a (now) old MPI implementation.  I expect a nontrivial amount of
> work would have to be done to get this to compile and run again, and
> that effort would probably be better served starting from scratch
> (using Todd Gamblin's wrap.py PMPI shim generator, for example).
> -
>
>
> > On Mar 21, 2018, at 2:23 AM, John Hearns via users <
> users@lists.open-mpi.org> wrote:
> >
> > Kaiming,  good luck with your project.  I think you should contact Barry
> Rountree directly. you will probably get good advice!
> >
> > It is worth saying that with Turboboost there is variation between each
> individual CPU die, even within the same SKU.
> > What Turboboost does is to set a thermal envelope, and the CPU core(s)
> ramp up in frequency till the thermal limit is reached.
> > So each CPU die is slightly different  (*)
> > Indeed in my last job we had a benchmarking exercise where the
> instruction was to explicitly turn off Turboboost.
> >
> >
> > (*) As I work at ASML I really should understand this better... I really
> should.
> >
> >
> >
> >
> >
> >
> > On 20 March 2018 at 19:34, Kaiming Ouyang  wrote:
> > Hi John,
> > Thank you for your advice. But this is only related to its
> functionality, and right now my problem is it cannot compile with new
> version openmpi.
> > The reason may come from its patch file since it needs to intercept MPI
> calls to profile some data. New version openmpi may change its framework so
> that this old software does not fit it anymore.
> >
> >
> > Kaiming Ouyang, Research Assistant.
> > Department of Computer Science and Engineering
> > University of California, Riverside
> > 900 University Avenue, Riverside, CA 92521
> >
> >
> > On Tue, Mar 20, 2018 at 10:46 AM, John Hearns via users <
> users@lists.open-mpi.org> wrote:
> > "It does not handle more recent improvements such as Intel's turbo
> > mode and the processor performance inhomogeneity that comes with it."
> > I guess it is easy enough to disable Turbo mode in the BIOS though.
> >
> >
> >
> > On 20 March 2018 at 17:48, Kaiming Ouyang  wrote:
> > I think the problem it has is it only deal with the old framework
> because it will intercept MPI calls and do some profiling. Here is the
> library:
> > https://github.com/LLNL/Adagio
> >
> > I checked the openmpi changelog. From openmpi 1.3, it began to switch to
> a new framework, and openmpi 1.4+ has different one too. This library only
> works under openmpi 1.2.
> > Thank you for your advise, I will try it. My current problem is this
> library seems to try to patch mpi.h file, but it fails during the patching
> process for new version openmpi. I don't know the reason yet, and will
> check it soon. Thank you.
> >
> > Kaiming Ouyang, Research Assistant.
> > Department of Computer Science and Engineering
> > University of California, Riverside
> > 900 University Avenue, Riverside, CA 92521
> >
> >
> > On Tue, Mar 20, 2018 at 4:35 AM, Jeff Squyres (jsquyres) <
> jsquy...@cisco.com> wrote:
> > On Mar 19, 2018, at 11:32 PM, Kaiming Ouyang  wrote:
> > >
> > > Thank you.
> > > I am using newest version HPL.
> > > I forgot to say I can run HPL with openmpi-3.0 under infiniband. The
> reason I want to use old version is I need to compile a library that only
> supports old version openmpi, so I am trying to do this tricky job.
> >
> > Gotcha.
> >
> > Is there something in particular about the old library that requires
> Open MPI v1.2.x?
> >
> > More specifically: is there a particular error you get when you try to
> use Open MPI v3.0.0 with that library?
> >
> > I ask because if the app supports the MPI API in Open MPI v1.2.9, then
> it also supports the MPI API in Open MPI v3.0.0.  We *have* changed lots of
> other things under the covers in that time, such as:
> >
> > - how those MPI API's are implemented
> > - mpirun (and friends) command line parameters
> > - MCA parameters
> > - compilation flags
> >
> > But many of those things might actually be mostly -- if not entirely --
> hidden from a library that uses MPI.
> >
> > My point: it may be easier to get your library to use a newer version of
> Open MPI than you think.  For example, if the library has some hard-coded
> flags in their 

Re: [OMPI users] Old version openmpi 1.2 support infiniband?

[Jeff Squyres (jsquyres)]

You might want to take that library author's advice from their README:

-
The source code herein was used as the basis of Rountree ICS 2009.  It
was my first nontrivial MPI tool and was never intended to be released
to the wider world.  I beleive it was tied rather tightly to a subset
of a (now) old MPI implementation.  I expect a nontrivial amount of 
work would have to be done to get this to compile and run again, and
that effort would probably be better served starting from scratch 
(using Todd Gamblin's wrap.py PMPI shim generator, for example).
-


> On Mar 21, 2018, at 2:23 AM, John Hearns via users  
> wrote:
> 
> Kaiming,  good luck with your project.  I think you should contact Barry 
> Rountree directly. you will probably get good advice!
> 
> It is worth saying that with Turboboost there is variation between each 
> individual CPU die, even within the same SKU.
> What Turboboost does is to set a thermal envelope, and the CPU core(s) ramp 
> up in frequency till the thermal limit is reached.
> So each CPU die is slightly different  (*)
> Indeed in my last job we had a benchmarking exercise where the instruction 
> was to explicitly turn off Turboboost.
> 
> 
> (*) As I work at ASML I really should understand this better... I really 
> should.
> 
> 
> 
> 
> 
> 
> On 20 March 2018 at 19:34, Kaiming Ouyang  wrote:
> Hi John,
> Thank you for your advice. But this is only related to its functionality, and 
> right now my problem is it cannot compile with new version openmpi. 
> The reason may come from its patch file since it needs to intercept MPI calls 
> to profile some data. New version openmpi may change its framework so that 
> this old software does not fit it anymore. 
> 
> 
> Kaiming Ouyang, Research Assistant.
> Department of Computer Science and Engineering
> University of California, Riverside
> 900 University Avenue, Riverside, CA 92521
> 
> 
> On Tue, Mar 20, 2018 at 10:46 AM, John Hearns via users 
>  wrote:
> "It does not handle more recent improvements such as Intel's turbo 
> mode and the processor performance inhomogeneity that comes with it."
> I guess it is easy enough to disable Turbo mode in the BIOS though.
> 
> 
> 
> On 20 March 2018 at 17:48, Kaiming Ouyang  wrote:
> I think the problem it has is it only deal with the old framework because it 
> will intercept MPI calls and do some profiling. Here is the library:
> https://github.com/LLNL/Adagio
> 
> I checked the openmpi changelog. From openmpi 1.3, it began to switch to a 
> new framework, and openmpi 1.4+ has different one too. This library only 
> works under openmpi 1.2.
> Thank you for your advise, I will try it. My current problem is this library 
> seems to try to patch mpi.h file, but it fails during the patching process 
> for new version openmpi. I don't know the reason yet, and will check it soon. 
> Thank you.
> 
> Kaiming Ouyang, Research Assistant.
> Department of Computer Science and Engineering
> University of California, Riverside
> 900 University Avenue, Riverside, CA 92521
> 
> 
> On Tue, Mar 20, 2018 at 4:35 AM, Jeff Squyres (jsquyres)  
> wrote:
> On Mar 19, 2018, at 11:32 PM, Kaiming Ouyang  wrote:
> >
> > Thank you.
> > I am using newest version HPL.
> > I forgot to say I can run HPL with openmpi-3.0 under infiniband. The reason 
> > I want to use old version is I need to compile a library that only supports 
> > old version openmpi, so I am trying to do this tricky job.
> 
> Gotcha.
> 
> Is there something in particular about the old library that requires Open MPI 
> v1.2.x?
> 
> More specifically: is there a particular error you get when you try to use 
> Open MPI v3.0.0 with that library?
> 
> I ask because if the app supports the MPI API in Open MPI v1.2.9, then it 
> also supports the MPI API in Open MPI v3.0.0.  We *have* changed lots of 
> other things under the covers in that time, such as:
> 
> - how those MPI API's are implemented
> - mpirun (and friends) command line parameters
> - MCA parameters
> - compilation flags
> 
> But many of those things might actually be mostly -- if not entirely -- 
> hidden from a library that uses MPI.
> 
> My point: it may be easier to get your library to use a newer version of Open 
> MPI than you think.  For example, if the library has some hard-coded flags in 
> their configure/Makefile to build with Open MPI, just replace those flags 
> with `mpicc --showme:BLAH` variants (see `mpicc --showme:help` for a full 
> listing).  This will have Open MPI tell you exactly what flags it needs to 
> compile, link, etc.
> 
> --
> Jeff Squyres
> jsquy...@cisco.com
> 
> ___
> users mailing list
> users@lists.open-mpi.org
> https://lists.open-mpi.org/mailman/listinfo/users
> 
> 
> ___
> users mailing list
> users@lists.open-mpi.org
> 

[hwloc-users] libhwloc soname change in 2.0.1rc1

[Brice Goglin]

Hello

In case you missed the announce yesterday, hwloc 2.0.1rc1 changes the
library soname from 12:0:0 to 15:0:0. On Linux, it means that we'll now
build libhwloc.so.15 instead of libhwloc.so.12. That means any
application built for hwloc 2.0.0 will need to be recompiled against 2.0.1.

I should have set the soname to 15:0:0 in 2.0.0 but I forgot. It may
cause issues because hwloc 1.11.x uses 12:x:y (we have "12" in both).
Given that 2.0.0 isn't widely used yet, I hope this way-too-late change
won't cause too many issues. Sorry.

As said on the download page, we want people to stop using 2.0.0 so that
we can forget this issue. If you already switched to hwloc 2.0.0 (and if
some applications are linked with libhwloc), please try to upgrade to
2.0.1 as soon as possible (final release expected next monday).

Brice

___
hwloc-users mailing list
hwloc-users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/hwloc-users

Re: [OMPI users] Old version openmpi 1.2 support infiniband?

[John Hearns via users]

Kaiming,  good luck with your project.  I think you should contact Barry
Rountree directly. you will probably get good advice!

It is worth saying that with Turboboost there is variation between each
individual CPU die, even within the same SKU.
What Turboboost does is to set a thermal envelope, and the CPU core(s) ramp
up in frequency till the thermal limit is reached.
So each CPU die is slightly different  (*)
Indeed in my last job we had a benchmarking exercise where the instruction
was to explicitly turn off Turboboost.


(*) As I work at ASML I really should understand this better... I really
should.






On 20 March 2018 at 19:34, Kaiming Ouyang  wrote:

> Hi John,
> Thank you for your advice. But this is only related to its functionality,
> and right now my problem is it cannot compile with new version openmpi.
> The reason may come from its patch file since it needs to intercept MPI
> calls to profile some data. New version openmpi may change its framework so
> that this old software does not fit it anymore.
>
>
> Kaiming Ouyang, Research Assistant.
> Department of Computer Science and Engineering
> University of California, Riverside
> 900 University Avenue, Riverside, CA 92521
>
>
> On Tue, Mar 20, 2018 at 10:46 AM, John Hearns via users <
> users@lists.open-mpi.org> wrote:
>
>> "It does not handle more recent improvements such as Intel's turbo
>> mode and the processor performance inhomogeneity that comes with it."
>> I guess it is easy enough to disable Turbo mode in the BIOS though.
>>
>>
>>
>> On 20 March 2018 at 17:48, Kaiming Ouyang  wrote:
>>
>>> I think the problem it has is it only deal with the old
>>> framework because it will intercept MPI calls and do some profiling. Here
>>> is the library:
>>> https://github.com/LLNL/Adagio
>>>
>>> I checked the openmpi changelog. From openmpi 1.3, it began to switch to
>>> a new framework, and openmpi 1.4+ has different one too. This library only
>>> works under openmpi 1.2.
>>> Thank you for your advise, I will try it. My current problem is this
>>> library seems to try to patch mpi.h file, but it fails during the patching
>>> process for new version openmpi. I don't know the reason yet, and will
>>> check it soon. Thank you.
>>>
>>> Kaiming Ouyang, Research Assistant.
>>> Department of Computer Science and Engineering
>>> University of California, Riverside
>>> 900 University Avenue, Riverside, CA 92521
>>>
>>>
>>> On Tue, Mar 20, 2018 at 4:35 AM, Jeff Squyres (jsquyres) <
>>> jsquy...@cisco.com> wrote:
>>>
 On Mar 19, 2018, at 11:32 PM, Kaiming Ouyang  wrote:
 >
 > Thank you.
 > I am using newest version HPL.
 > I forgot to say I can run HPL with openmpi-3.0 under infiniband. The
 reason I want to use old version is I need to compile a library that only
 supports old version openmpi, so I am trying to do this tricky job.

 Gotcha.

 Is there something in particular about the old library that requires
 Open MPI v1.2.x?

 More specifically: is there a particular error you get when you try to
 use Open MPI v3.0.0 with that library?

 I ask because if the app supports the MPI API in Open MPI v1.2.9, then
 it also supports the MPI API in Open MPI v3.0.0.  We *have* changed lots of
 other things under the covers in that time, such as:

 - how those MPI API's are implemented
 - mpirun (and friends) command line parameters
 - MCA parameters
 - compilation flags

 But many of those things might actually be mostly -- if not entirely --
 hidden from a library that uses MPI.

 My point: it may be easier to get your library to use a newer version
 of Open MPI than you think.  For example, if the library has some
 hard-coded flags in their configure/Makefile to build with Open MPI, just
 replace those flags with `mpicc --showme:BLAH` variants (see `mpicc
 --showme:help` for a full listing).  This will have Open MPI tell you
 exactly what flags it needs to compile, link, etc.

 --
 Jeff Squyres
 jsquy...@cisco.com

 ___
 users mailing list
 users@lists.open-mpi.org
 https://lists.open-mpi.org/mailman/listinfo/users

>>>
>>>
>>> ___
>>> users mailing list
>>> users@lists.open-mpi.org
>>> https://lists.open-mpi.org/mailman/listinfo/users
>>>
>>
>>
>> ___
>> users mailing list
>> users@lists.open-mpi.org
>> https://lists.open-mpi.org/mailman/listinfo/users
>>
>
>
> ___
> users mailing list
> users@lists.open-mpi.org
> https://lists.open-mpi.org/mailman/listinfo/users
>
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users