Re: [hwloc-users] Process and thread binding

[Samuel Thibault]

Gabriele Fatigati, le Mon 12 Sep 2011 15:50:45 +0200, a écrit :
> thanks very much for your explanations. But I don't understand why a process
> inherits core bound of his threads

On Linux, there is no such thing as "process binding", only "thread
binding". hwloc emulates the former by using the latter.

Samuel

Re: [hwloc-users] Process and thread binding

[Gabriele Fatigati]

Ok Brice,

thanks very much for your explanations. But I don't understand why a process
inherits core bound of his threads according to your example:

>It worked because you never mixed it with single thread binding. If you
bind process X to >coreA and then thread Y of process X to coreB, what you
should now see with get_cpubind is >that process X is now bound to cores
A+B, thread Y to B, and all other threads to A.




2011/9/12 Brice Goglin 

> Le 12/09/2011 14:17, Gabriele Fatigati a écrit :
> > Mm, and why? In a hybrid code ( MPI + OpenMP), my idea is to bind a
> > single MPI process in  one core, and his threads in other cores.
> > Otherwise I have all threads that runs on a single core.
> >
>
> The usual way to do that is to first bind the entire process to all
> cores available to all its thread and then bind each thread to a single
> core.
>
> For instance, if doing Open MPI + OpenMP with one process per socket
> * you pass --bind-to-socket on the mpirun/mpiexec command-line
> * when the MPI process starts, the OpenMP runtime calls something like
> get_cpubind to find out how many cores were given to it
> * it creates the exact same number of OpenMP threads and bind one of
> them on each core
>
> Brice
>
> ___
> hwloc-users mailing list
> hwloc-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users
>



-- 
Ing. Gabriele Fatigati

HPC specialist

SuperComputing Applications and Innovation Department

Via Magnanelli 6/3, Casalecchio di Reno (BO) Italy

www.cineca.itTel:   +39 051 6171722

g.fatigati [AT] cineca.it

Re: [hwloc-users] Process and thread binding

[Brice Goglin]

Le 12/09/2011 14:17, Gabriele Fatigati a écrit :
> Mm, and why? In a hybrid code ( MPI + OpenMP), my idea is to bind a
> single MPI process in  one core, and his threads in other cores.
> Otherwise I have all threads that runs on a single core.
>

The usual way to do that is to first bind the entire process to all
cores available to all its thread and then bind each thread to a single
core.

For instance, if doing Open MPI + OpenMP with one process per socket
* you pass --bind-to-socket on the mpirun/mpiexec command-line
* when the MPI process starts, the OpenMP runtime calls something like
get_cpubind to find out how many cores were given to it
* it creates the exact same number of OpenMP threads and bind one of
them on each core

Brice

Re: [hwloc-users] Process and thread binding

[Gabriele Fatigati]

Ok,

>A process is a container that contains one or several threads. The binding
is where >something can run. It's normal that "where a process can run" is
"where any of its >threads can run", which means it's the logical OR of
their binding.


Ok, now it's clear.

I'm using hwloc_get_last_cpu_location(), not get_cpubind(), sorry..

Well, the first thing to understand is what you're trying to do here. In the
vast majority of cases, binding a thread outside of the binding of the
corresponding process looks like buggy programming.


Mm, and why? In a hybrid code ( MPI + OpenMP), my idea is to bind a single
MPI process in  one core, and his threads in other cores. Otherwise I have
all threads that runs on a single core..


2011/9/12 Brice Goglin 

> **
> Le 12/09/2011 13:58, Gabriele Fatigati a écrit :
>
> Hi Brice,
>
>  but in the manual is not written that get_cpubind() returns the logical
> OR of the binding of all threads... I ever understand that returns the
> bind of the calloer, where the caller can be process or thread..
>
>
> A process is a container that contains one or several threads. The binding
> is where something can run. It's normal that "where a process can run" is
> "where any of its threads can run", which means it's the logical OR of their
> binding.
>
>
>
>
>  I'm mixing bind of process and threads, and I've noted that if the
> process and thread are on the same NUMA node, works well, also on different
> cores.
>
>  If the NUMA node of process is different of NUMA node of threads, there
> is a problem.
>
>
> Well, the first thing to understand is what you're trying to do here. In
> the vast majority of cases, binding a thread outside of the binding of the
> corresponding process looks like buggy programming.
>
> Brice
>
>
> ___
> hwloc-users mailing list
> hwloc-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users
>



-- 
Ing. Gabriele Fatigati

HPC specialist

SuperComputing Applications and Innovation Department

Via Magnanelli 6/3, Casalecchio di Reno (BO) Italy

www.cineca.itTel:   +39 051 6171722

g.fatigati [AT] cineca.it

Re: [hwloc-users] Process and thread binding

[Gabriele Fatigati]

Hi Brice,

but in the manual is not written that get_cpubind() returns the logical OR
of the binding of all threads... I ever understand that returns the bind of
the calloer, where the caller can be process or thread..

I'm mixing bind of process and threads, and I've noted that if the process
and thread are on the same NUMA node, works well, also on different cores.

If the NUMA node of process is different of NUMA node of threads, there is a
problem.



2011/9/12 Brice Goglin 

> **
> Le 12/09/2011 13:29, Gabriele Fatigati a écrit :
>
> Hi Birce,
>
>  I'm so  confused..
>
>  I'm binding MPI processes with set_cpu_bind and it works well. The
> problem is when I  try to bind process and threads.
>
>  It seem that thread process influence bind of main thread.
>
>  And from hwloc manual:
>
>
>  hwloc_set_cpubind()
>
>  Bind *current process* or thread on cpus given in bitmap set.
>
>  Why you are saying tha process bind is not possible? I'm using it and it
> work well!
>
>
> It worked because you never mixed it with single thread binding. If you
> bind process X to coreA and then thread Y of process X to coreB, what you
> should now see with get_cpubind is that process X is now bound to cores A+B,
> thread Y to B, and all other threads to A.
>
> Brice
>
>
> ___
> hwloc-users mailing list
> hwloc-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users
>



-- 
Ing. Gabriele Fatigati

HPC specialist

SuperComputing Applications and Innovation Department

Via Magnanelli 6/3, Casalecchio di Reno (BO) Italy

www.cineca.itTel:   +39 051 6171722

g.fatigati [AT] cineca.it

Re: [hwloc-users] Process and thread binding

[Brice Goglin]

Le 12/09/2011 13:29, Gabriele Fatigati a écrit :
> Hi Birce,
>
> I'm so  confused..
>
> I'm binding MPI processes with set_cpu_bind and it works well. The
> problem is when I  try to bind process and threads.
>
> It seem that thread process influence bind of main thread.
>
> And from hwloc manual:
>
>
> hwloc_set_cpubind()
>
> Bind *current process* or thread on cpus given in bitmap set.
>
> Why you are saying tha process bind is not possible? I'm using it and
> it work well!

It worked because you never mixed it with single thread binding. If you
bind process X to coreA and then thread Y of process X to coreB, what
you should now see with get_cpubind is that process X is now bound to
cores A+B, thread Y to B, and all other threads to A.

Brice