Re: [hwloc-users] Process and thread binding

2011-09-12 Thread Samuel Thibault
Gabriele Fatigati, le Mon 12 Sep 2011 15:50:45 +0200, a écrit :
> thanks very much for your explanations. But I don't understand why a process
> inherits core bound of his threads

On Linux, there is no such thing as "process binding", only "thread
binding". hwloc emulates the former by using the latter.

Samuel


Re: [hwloc-users] Process and thread binding

2011-09-12 Thread Gabriele Fatigati
Ok Brice,

thanks very much for your explanations. But I don't understand why a process
inherits core bound of his threads according to your example:

>It worked because you never mixed it with single thread binding. If you
bind process X to >coreA and then thread Y of process X to coreB, what you
should now see with get_cpubind is >that process X is now bound to cores
A+B, thread Y to B, and all other threads to A.




2011/9/12 Brice Goglin 

> Le 12/09/2011 14:17, Gabriele Fatigati a écrit :
> > Mm, and why? In a hybrid code ( MPI + OpenMP), my idea is to bind a
> > single MPI process in  one core, and his threads in other cores.
> > Otherwise I have all threads that runs on a single core.
> >
>
> The usual way to do that is to first bind the entire process to all
> cores available to all its thread and then bind each thread to a single
> core.
>
> For instance, if doing Open MPI + OpenMP with one process per socket
> * you pass --bind-to-socket on the mpirun/mpiexec command-line
> * when the MPI process starts, the OpenMP runtime calls something like
> get_cpubind to find out how many cores were given to it
> * it creates the exact same number of OpenMP threads and bind one of
> them on each core
>
> Brice
>
> ___
> hwloc-users mailing list
> hwloc-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users
>



-- 
Ing. Gabriele Fatigati

HPC specialist

SuperComputing Applications and Innovation Department

Via Magnanelli 6/3, Casalecchio di Reno (BO) Italy

www.cineca.itTel:   +39 051 6171722

g.fatigati [AT] cineca.it


Re: [hwloc-users] Process and thread binding

2011-09-12 Thread Brice Goglin
Le 12/09/2011 14:17, Gabriele Fatigati a écrit :
> Mm, and why? In a hybrid code ( MPI + OpenMP), my idea is to bind a
> single MPI process in  one core, and his threads in other cores.
> Otherwise I have all threads that runs on a single core.
>

The usual way to do that is to first bind the entire process to all
cores available to all its thread and then bind each thread to a single
core.

For instance, if doing Open MPI + OpenMP with one process per socket
* you pass --bind-to-socket on the mpirun/mpiexec command-line
* when the MPI process starts, the OpenMP runtime calls something like
get_cpubind to find out how many cores were given to it
* it creates the exact same number of OpenMP threads and bind one of
them on each core

Brice



Re: [hwloc-users] Process and thread binding

2011-09-12 Thread Gabriele Fatigati
Ok,

>A process is a container that contains one or several threads. The binding
is where >something can run. It's normal that "where a process can run" is
"where any of its >threads can run", which means it's the logical OR of
their binding.


Ok, now it's clear.

I'm using hwloc_get_last_cpu_location(), not get_cpubind(), sorry..

Well, the first thing to understand is what you're trying to do here. In the
vast majority of cases, binding a thread outside of the binding of the
corresponding process looks like buggy programming.


Mm, and why? In a hybrid code ( MPI + OpenMP), my idea is to bind a single
MPI process in  one core, and his threads in other cores. Otherwise I have
all threads that runs on a single core..


2011/9/12 Brice Goglin 

> **
> Le 12/09/2011 13:58, Gabriele Fatigati a écrit :
>
> Hi Brice,
>
>  but in the manual is not written that get_cpubind() returns the logical
> OR of the binding of all threads... I ever understand that returns the
> bind of the calloer, where the caller can be process or thread..
>
>
> A process is a container that contains one or several threads. The binding
> is where something can run. It's normal that "where a process can run" is
> "where any of its threads can run", which means it's the logical OR of their
> binding.
>
>
>
>
>  I'm mixing bind of process and threads, and I've noted that if the
> process and thread are on the same NUMA node, works well, also on different
> cores.
>
>  If the NUMA node of process is different of NUMA node of threads, there
> is a problem.
>
>
> Well, the first thing to understand is what you're trying to do here. In
> the vast majority of cases, binding a thread outside of the binding of the
> corresponding process looks like buggy programming.
>
> Brice
>
>
> ___
> hwloc-users mailing list
> hwloc-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users
>



-- 
Ing. Gabriele Fatigati

HPC specialist

SuperComputing Applications and Innovation Department

Via Magnanelli 6/3, Casalecchio di Reno (BO) Italy

www.cineca.itTel:   +39 051 6171722

g.fatigati [AT] cineca.it


Re: [hwloc-users] Process and thread binding

2011-09-12 Thread Brice Goglin
Le 12/09/2011 13:58, Gabriele Fatigati a écrit :
> Hi Brice,
>
> but in the manual is not written that get_cpubind() returns the
> logical OR of the binding of all threads... I ever understand that
> returns the bind of the calloer, where the caller can be process or
> thread..

A process is a container that contains one or several threads. The
binding is where something can run. It's normal that "where a process
can run" is "where any of its threads can run", which means it's the
logical OR of their binding.


>
> I'm mixing bind of process and threads, and I've noted that if the
> process and thread are on the same NUMA node, works well, also on
> different cores.
>
> If the NUMA node of process is different of NUMA node of threads,
> there is a problem.
>

Well, the first thing to understand is what you're trying to do here. In
the vast majority of cases, binding a thread outside of the binding of
the corresponding process looks like buggy programming.

Brice



Re: [hwloc-users] Process and thread binding

2011-09-12 Thread Gabriele Fatigati
Hi Brice,

but in the manual is not written that get_cpubind() returns the logical OR
of the binding of all threads... I ever understand that returns the bind of
the calloer, where the caller can be process or thread..

I'm mixing bind of process and threads, and I've noted that if the process
and thread are on the same NUMA node, works well, also on different cores.

If the NUMA node of process is different of NUMA node of threads, there is a
problem.



2011/9/12 Brice Goglin 

> **
> Le 12/09/2011 13:29, Gabriele Fatigati a écrit :
>
> Hi Birce,
>
>  I'm so  confused..
>
>  I'm binding MPI processes with set_cpu_bind and it works well. The
> problem is when I  try to bind process and threads.
>
>  It seem that thread process influence bind of main thread.
>
>  And from hwloc manual:
>
>
>  hwloc_set_cpubind()
>
>  Bind *current process* or thread on cpus given in bitmap set.
>
>  Why you are saying tha process bind is not possible? I'm using it and it
> work well!
>
>
> It worked because you never mixed it with single thread binding. If you
> bind process X to coreA and then thread Y of process X to coreB, what you
> should now see with get_cpubind is that process X is now bound to cores A+B,
> thread Y to B, and all other threads to A.
>
> Brice
>
>
> ___
> hwloc-users mailing list
> hwloc-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users
>



-- 
Ing. Gabriele Fatigati

HPC specialist

SuperComputing Applications and Innovation Department

Via Magnanelli 6/3, Casalecchio di Reno (BO) Italy

www.cineca.itTel:   +39 051 6171722

g.fatigati [AT] cineca.it


Re: [hwloc-users] Process and thread binding

2011-09-12 Thread Brice Goglin
Le 12/09/2011 13:29, Gabriele Fatigati a écrit :
> Hi Birce,
>
> I'm so  confused..
>
> I'm binding MPI processes with set_cpu_bind and it works well. The
> problem is when I  try to bind process and threads.
>
> It seem that thread process influence bind of main thread.
>
> And from hwloc manual:
>
>
> hwloc_set_cpubind()
>
> Bind *current process* or thread on cpus given in bitmap set.
>
> Why you are saying tha process bind is not possible? I'm using it and
> it work well!

It worked because you never mixed it with single thread binding. If you
bind process X to coreA and then thread Y of process X to coreB, what
you should now see with get_cpubind is that process X is now bound to
cores A+B, thread Y to B, and all other threads to A.

Brice



Re: [hwloc-users] Process and thread binding

2011-09-12 Thread Brice Goglin
Le 12/09/2011 12:52, Gabriele Fatigati a écrit :
> Dear hwloc users,
>
> I'm binding process in a NUMA node and also associated OpenMP threads.
> I've noted that, if I bind  execution  of all on different cores in
> the same NUMA node , it works well.
>
> If I bind process in NUMA node 0 for example, and thread on NUMA node
> 1, the process is bound on NUMA node 1, and not 0.
>
> Why this? Thread binding influence bind of main process?
>

Hello Gabriele,
At least on Linux, you cannot bind a process. When you ask hwloc to bind
a process, it just binds all threads contained in this process. When you
do get_cpubind on a process, it returns the logical OR of the binding of
all threads.
Brice