Re: [hwloc-devel] Strange difference

[Brice Goglin]

Yo Ralph,
The reason is that get_type_depth() returns negative numbers for errors
(when the type doesn't exist or exists multiple times in the topology).
topology_get_depth() never fails, so always returns >=0.
And indeed that's annoying when the compiler is picky.
Brice



Le 12/10/2013 20:37, Ralph Castain a écrit :
> Yo guys
>
> I was doing some work that involved traversing the hwloc topo tree, and 
> encountered the following odd discrepancy.
>
> hwloc_topology_get_depth  => returns "unsigned"
>
> hwloc_get_type_depth  => returns "int"
>
> Why the difference? Makes it hard sometimes to avoid the "comparison between 
> unsigned and signed" warnings when using these functions.
> Ralph
>
> ___
> hwloc-devel mailing list
> hwloc-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel

[hwloc-devel] Strange difference

[Ralph Castain]

Yo guys

I was doing some work that involved traversing the hwloc topo tree, and 
encountered the following odd discrepancy.

hwloc_topology_get_depth  => returns "unsigned"

hwloc_get_type_depth  => returns "int"

Why the difference? Makes it hard sometimes to avoid the "comparison between 
unsigned and signed" warnings when using these functions.
Ralph

Re: [hwloc-devel] Strange difference

[Samuel Thibault]

Brice Goglin, le Wed 31 Mar 2010 11:37:17 +0200, a écrit :
> We might need to replace some occurences of "logical processor" in the
> doc with "processing unit". Or use both from time to time to make it
> clear that it's very similar (and explain the difference somewhere).

I'd say keep it in the glossary section of the documentation, to make it
clear that by PU we mean "logical processor", and not "core".

Samuel

Re: [hwloc-devel] Strange difference

[Brice Goglin]

Jeff Squyres wrote:
> Is it a crime to use the full word "Processor"?  At least on my machine, the 
> output width is still far less than 80 characters, so the full word should be 
> no problem.  But I don't know if there are other strange topologies out there 
> that would take up more space...?  (it certainly would in the graphic 
> output...)
>   

Trying to summarize what was said:
* There is no output width problem in text mode
* There is a strong width problem in graphic mode
* "Processor": ambiguous wrt socket/die, and needs a much shorter
version in the graphic output
* "CPU": ambiguous wrt socket/die
* "Proc": ambiguous wrt socket/die, and also with Process
* "PU": beware that it'll need some thinking since we cannot bind
processes/threads on GPU and SPU. Could be an empty CPU set. Or another
PU object.

Anybody really against PU ? Otherwise I'll commit the attached patch and
maybe we'll finally release 1.0-rc1 one day :)

The patch renames PROC into PU, both in the core and tools. I added some
backward compat code so that the "proc" string is still converted into
HWLOC_OBJ_PU (but it's not documented).

We might need to replace some occurences of "logical processor" in the
doc with "processing unit". Or use both from time to time to make it
clear that it's very similar (and explain the difference somewhere).

Brice

diff --git a/NEWS b/NEWS
index 7c305e8..83b577a 100644
--- a/NEWS
+++ b/NEWS
@@ -25,6 +25,8 @@ Version 1.0.0
   + Support binding of entire multithreaded process on Linux.
   + Cleanup XML export/import.
 * Objects
+  + HWLOC_OBJ_PROC is renamed into HWLOC_OBJ_PU for "Processing Unit",
+its stringified type name is now "PU".
   + Rework memory attributes.
   + Add different cpusets in each object to specify processors that
 are offline, unavailable, ...
diff --git a/doc/Makefile.am b/doc/Makefile.am
index 0a3537d..fe521a0 100644
--- a/doc/Makefile.am
+++ b/doc/Makefile.am
@@ -233,7 +233,7 @@ man3_MANS = \
 $(DOX_MAN_DIR)/man3/HWLOC_OBJ_MACHINE.3 \
 $(DOX_MAN_DIR)/man3/HWLOC_OBJ_MISC.3 \
 $(DOX_MAN_DIR)/man3/HWLOC_OBJ_NODE.3 \
-$(DOX_MAN_DIR)/man3/HWLOC_OBJ_PROC.3 \
+$(DOX_MAN_DIR)/man3/HWLOC_OBJ_PU.3 \
 $(DOX_MAN_DIR)/man3/HWLOC_OBJ_SOCKET.3 \
 $(DOX_MAN_DIR)/man3/HWLOC_OBJ_SYSTEM.3 \
 $(DOX_MAN_DIR)/man3/HWLOC_TOPOLOGY_FLAG_WHOLE_SYSTEM.3 \
diff --git a/doc/hwloc-hello.c b/doc/hwloc-hello.c
index a86046d..95586a7 100644
--- a/doc/hwloc-hello.c
+++ b/doc/hwloc-hello.c
@@ -84,7 +84,7 @@ int main(void)
  */
 levels = 0;
 size = 0;
-for (obj = hwloc_get_obj_by_type(topology, HWLOC_OBJ_PROC, 0);
+for (obj = hwloc_get_obj_by_type(topology, HWLOC_OBJ_PU, 0);
  obj;
  obj = obj->parent)
   if (obj->type == HWLOC_OBJ_CACHE) {
diff --git a/doc/hwloc.doxy b/doc/hwloc.doxy
index c2601b6..adbfff9 100644
--- a/doc/hwloc.doxy
+++ b/doc/hwloc.doxy
@@ -769,7 +769,7 @@ os_index field of objects, but such practice should be avoided as much
 as possible for the reasons described above (except perhaps for
 prettyprinting / debugging purposes).

-::HWLOC_OBJ_PROC objects are supposed to have different physical/OS
+::HWLOC_OBJ_PU objects are supposed to have different physical/OS
 indexes since the OS uses them for binding.  The \p os_index field of
 these objects provides the identifier that may be used for such
 binding, and hwloc_get_proc_obj_by_os_index() finds the object
diff --git a/include/hwloc.h b/include/hwloc.h
index 1e519a5..fc8a278 100644
--- a/include/hwloc.h
+++ b/include/hwloc.h
@@ -94,7 +94,7 @@ typedef enum {
 			  * A computation unit (may be shared by several
 			  * logical processors).
 			  */
-  HWLOC_OBJ_PROC,	/**< \brief (Logical) Processor.
+  HWLOC_OBJ_PU,		/**< \brief Processing Unit, or (Logical) Processor.
 			  * An execution unit (may share a core with some
 			  * other logical processors, e.g. in the case of
 			  * an SMT core).
@@ -124,7 +124,7 @@ typedef enum {
  * contain sockets which contain caches, which contain cores, which contain
  * processors).
  *
- * \note HWLOC_OBJ_PROC will always be the deepest.
+ * \note HWLOC_OBJ_PU will always be the deepest.
  * \note This does not mean that the actual topology will respect that order:
  * e.g. as of today cores may also contain caches, and sockets may also contain
  * nodes. This is thus just to be seen as a fallback comparison method.
@@ -373,7 +373,7 @@ HWLOC_DECLSPEC void hwloc_topology_check(hwloc_topology_t topology);
 /** \brief Ignore an object type.
  *
  * Ignore all objects from the given type.
- * The bottom-level type HWLOC_OBJ_PROC may not be ignored.
+ * The bottom-level type HWLOC_OBJ_PU may not be ignored.
  * The top-level object of the hierarchy will never be ignored, even if this function
  * succeeds.
  */
@@ -383,7 +383,7 @@ HWLOC_DECLSPEC int hwloc_topology_ignore_type(hwloc_topology_t topology, hwloc_o
  *
  * Ignore all objects from the given type as long as they do not bring any 

Re: [hwloc-devel] Strange difference

[Jeff Squyres (jsquyres)]

True (processor can be ambiguous), but remember that we're talking about 
sockets, cores, and hardware threads. 

-jms
Sent from my PDA.  No type good.

- Original Message -
From: hwloc-devel-boun...@open-mpi.org <hwloc-devel-boun...@open-mpi.org>
To: hwloc-de...@open-mpi.org <hwloc-de...@open-mpi.org>
Sent: Tue Mar 30 05:40:30 2010
Subject: Re: [hwloc-devel] Strange difference

On Sat, 27 Mar 2010 08:23:52 am Jeff Squyres wrote:

> I think that we need to differentiate between the different meanings of
> "CPU" here -- CPU could also mean "socket", for example...

...so can processor. :-(Really the only unambiguous way I've come across 
(so far) describing things is in terms of sockets and cores.

Wouldn't be surprised if someone pointed out an ambiguity in those too!

cheers,
Chris
-- 
 Chris Samuel  :  http://www.csamuel.org/  :  Melbourne, VIC

This email may come with a PGP signature as a file. Do not panic.
For more info see: http://en.wikipedia.org/wiki/OpenPGP

Re: [hwloc-devel] Strange difference

[Brice Goglin]

Chris Samuel wrote:
> On Sat, 27 Mar 2010 05:15:50 am Jeff Squyres wrote:
>
>   
>> At least on my machine, the output width is still far less than 80
>> characters, so the full word should be no problem.  But I don't know if
>> there are other strange topologies out there that would take up more
>> space...?
>> 
>
> Be interesting to see what it would look like on a large SGI UV, for 
> instance..
>   

On a old Altix 4700 with 256 itanium cores, it's still far from 80 columns:

Machine (493GB)
  Misc1 #0 (123GB)
Misc0 #0 (31GB)
  NUMANode #0 (phys=0 7875MB)
Socket #0
  Core #0 + P #0 (phys=0)
  Core #1 + P #1 (phys=1)
Socket #1
  Core #2 + P #2 (phys=2)
  [...]


And in verbose mode:

Machine (phys=0 total=516912176KB)
  Misc1 #0 (total=129224048KB)
Misc0 #0 (total=32296336KB)
  NUMANode #0 (phys=0 local=8064400KB total=8064400KB)
Socket #0 (phys=0)
  Core #0 (phys=0)
P #0 (phys=0)
  Core #1 (phys=1)
P #1 (phys=1)


(generated from sysfs tarballs, but shouldn't be different from the
actual machine)

On a small Altix UV with 24 Nehalem-EX cores with HT, the width is similar.

IIRC, the largest UV is 2048 cores, the hierarchy should be
* 1 machine
* 256 NUMA nodes grouped in 3 or 4 levels of "misc" objects depending on
their distances
* in each NUMA Node, 1 socket x 1 L3 x 8 L2 x 1 L1 x 1 core x 2 threads
That's 12 levels of hierarchy, which means 22 characters of indentation
in the lstopo output. Each line is usually 10-20 characters, except when
you have some memory (it could be 40-50 characters in this case). So we
should be ok.

Brice

Re: [hwloc-devel] Strange difference

[Chris Samuel]

On Sat, 27 Mar 2010 08:23:52 am Jeff Squyres wrote:

> I think that we need to differentiate between the different meanings of
> "CPU" here -- CPU could also mean "socket", for example...

...so can processor. :-(Really the only unambiguous way I've come across 
(so far) describing things is in terms of sockets and cores.

Wouldn't be surprised if someone pointed out an ambiguity in those too!

cheers,
Chris
-- 
 Chris Samuel  :  http://www.csamuel.org/  :  Melbourne, VIC

This email may come with a PGP signature as a file. Do not panic.
For more info see: http://en.wikipedia.org/wiki/OpenPGP


signature.asc
Description: This is a digitally signed message part.

Re: [hwloc-devel] Strange difference

[Bert Wesarg]

On Sat, Mar 27, 2010 at 01:50, Jeff Squyres  wrote:
> On Mar 26, 2010, at 5:35 PM, Brice Goglin wrote:
>
>> > Fair enough.  How about still just keeping "P" in the graphic output, 
>> > then?  But "processor" in the prettyprint?
>>
>> IIRC, somebody said "PU" (for "processing unit") could be a good
>> solution. Otherwise, I am ok with "Proc" or "Processor", with a small
>> preference for the former.
>
> I think I still am uncomfortable with "proc" because it's too much like 
> "process".  But that could be just me.
>
> PU might be suitable.
>
>> By the way, this is also what hwloc_type_string() would return. Unless
>> we keep it unchanged and just hack lstopo to use its own stringified
>> type name ?
>
> I wouldn't mind the hack (too much), but it does seem a little inelegant.  If 
> we hate everything else, let's settle on "PU".

"PU" may also be a little future safe, when GPU cores become common.

Bert

>
> --
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/
>
>
> ___
> hwloc-devel mailing list
> hwloc-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel
>

Re: [hwloc-devel] Strange difference

[Jeff Squyres]

On Mar 26, 2010, at 5:35 PM, Brice Goglin wrote:

> > Fair enough.  How about still just keeping "P" in the graphic output, then? 
> >  But "processor" in the prettyprint?
> 
> IIRC, somebody said "PU" (for "processing unit") could be a good
> solution. Otherwise, I am ok with "Proc" or "Processor", with a small
> preference for the former.

I think I still am uncomfortable with "proc" because it's too much like 
"process".  But that could be just me.

PU might be suitable.

> By the way, this is also what hwloc_type_string() would return. Unless
> we keep it unchanged and just hack lstopo to use its own stringified
> type name ?

I wouldn't mind the hack (too much), but it does seem a little inelegant.  If 
we hate everything else, let's settle on "PU".

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [hwloc-devel] Strange difference

[Brice Goglin]

Jeff Squyres wrote:
> On Mar 26, 2010, at 5:20 PM, Samuel Thibault wrote:
>
>   
>> That's still very large. We are going toward dozens of cores on each
>> sockets, we really need to keep them small :)
>> 
>
> Fair enough.  How about still just keeping "P" in the graphic output, then?  
> But "processor" in the prettyprint?
>   


IIRC, somebody said "PU" (for "processing unit") could be a good
solution. Otherwise, I am ok with "Proc" or "Processor", with a small
preference for the former.

By the way, this is also what hwloc_type_string() would return. Unless
we keep it unchanged and just hack lstopo to use its own stringified
type name ?

Brice

Re: [hwloc-devel] Strange difference

[Samuel Thibault]

Jeff Squyres, le Fri 26 Mar 2010 17:05:39 -0400, a écrit :
> On Mar 26, 2010, at 4:16 PM, Samuel Thibault wrote:
> 
> > > Is it a crime to use the full word "Processor"?  At least on my machine, 
> > > the output width is still far less than 80 characters, so the full word 
> > > should be no problem.  But I don't know if there are other strange 
> > > topologies out there that would take up more space...?  (it certainly 
> > > would in the graphic output...)
> > 
> > The graphical output is a reason to at least have a function that
> > returns P, yes, else the output is very large.
> 
> How about "Processor" in the text outputs, and a shorter version in the 
> graphic output?
> 
> Shorten "processor" to:
> 
> Prcssr
> Prcsr
> Procr
> Procsr

That's still very large. We are going toward dozens of cores on each
sockets, we really need to keep them small :)

Samuel

Re: [hwloc-devel] Strange difference

[Bert Wesarg]

On Fri, Mar 26, 2010 at 22:05, Jeff Squyres  wrote:
> On Mar 26, 2010, at 4:16 PM, Samuel Thibault wrote:
>
>> > Is it a crime to use the full word "Processor"?  At least on my machine, 
>> > the output width is still far less than 80 characters, so the full word 
>> > should be no problem.  But I don't know if there are other strange 
>> > topologies out there that would take up more space...?  (it certainly 
>> > would in the graphic output...)
>>
>> The graphical output is a reason to at least have a function that
>> returns P, yes, else the output is very large.
>
> How about "Processor" in the text outputs, and a shorter version in the 
> graphic output?
>
> Shorten "processor" to:
>
> Prcssr
> Prcsr
> Procr
> Procsr
> ...?

For htop I use just "CPU".

Bert

Re: [hwloc-devel] Strange difference

[Samuel Thibault]

Jeff Squyres, le Fri 26 Mar 2010 14:15:50 -0400, a écrit :
> On Mar 26, 2010, at 2:01 PM, Brice Goglin wrote:
> 
> > > I like "Proc" instead of "P" even for the non-v output.  :-)
> > 
> > I am not against it, but I don't remember the reason for the initial
> > change. Maybe because "processor" is confusing for some people (logical
> > vs physical socket) ?
> 
> Oh crap.  I think you're right.  And I think I even asked for that.  ;-)
> 
> Hmm.  
> 
> Is it a crime to use the full word "Processor"?  At least on my machine, the 
> output width is still far less than 80 characters, so the full word should be 
> no problem.  But I don't know if there are other strange topologies out there 
> that would take up more space...?  (it certainly would in the graphic 
> output...)

The graphical output is a reason to at least have a function that
returns P, yes, else the output is very large.

Samuel

Re: [hwloc-devel] Strange difference

[Jeff Squyres]

On Mar 26, 2010, at 1:25 PM, Brice Goglin wrote:

> The reason was that the "phys" attribute is mostly useless for anything
> but Proc and NUMANode.
> 
> Current other differences when adding -v are:
> * "Proc" if -v instead of "P"

I like "Proc" instead of "P" even for the non-v output.  :-)

> * print local memory size

Is the Machine memory size supposed to agree between the two outputs, or does 
the -v output roll up some sizes together?

Machine (3945MB)
Machine (phys=0 local=4039640KB total=4039640KB)

I'm obviously good with -v having more information.  I guess my reaction was 
because it wasn't just *more* information -- the *same* information was in a 
different format, and that struck me as weird.

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [hwloc-devel] Strange difference

[Samuel Thibault]

Jeff Squyres, le Fri 26 Mar 2010 10:47:15 -0400, a écrit :
> The output of "lstopo -l" is different than "lstopo -l -v" -- is that 
> intentional?

Well, yes, it is supposed to display less information :)
Which precise difference are you referring to?

Samuel

> -
> [7:45] svbu-mpi:~/svn/hwloc % ./utils/lstopo -l 
> Machine (3945MB)
>   Socket #0
> L2 #0 (2048KB) + L1 #0 (16KB) + Core #0
>   P #0 (phys=0)
>   P #1 (phys=4)
> L2 #1 (2048KB) + L1 #1 (16KB) + Core #1
>   P #2 (phys=2)
>   P #3 (phys=6)
>   Socket #1
> L2 #2 (2048KB) + L1 #2 (16KB) + Core #2
>   P #4 (phys=1)
>   P #5 (phys=5)
> L2 #3 (2048KB) + L1 #3 (16KB) + Core #3
>   P #6 (phys=3)
>   P #7 (phys=7)
> -
> 
> -
> [7:45] svbu-mpi:~/svn/hwloc % ./utils/lstopo -l -v
> Machine (phys=0 local=4039640KB total=4039640KB)
>   Socket #0 (phys=0)
> L2Cache #0 (2048KB)
>   L1Cache #0 (16KB)
> Core #0 (phys=0)
>   P #0 (phys=0)
>   P #1 (phys=4)
> L2Cache #1 (2048KB)
>   L1Cache #1 (16KB)
> Core #1 (phys=1)
>   P #2 (phys=2)
>   P #3 (phys=6)
>   Socket #1 (phys=1)
> L2Cache #2 (2048KB)
>   L1Cache #2 (16KB)
> Core #2 (phys=0)
>   P #4 (phys=1)
>   P #5 (phys=5)
> L2Cache #3 (2048KB)
>   L1Cache #3 (16KB)
> Core #3 (phys=1)
>   P #6 (phys=3)
>   P #7 (phys=7)
> depth 0:  1 Machine (type #1)
>  depth 1: 2 Sockets (type #3)
>   depth 2:4 Caches (type #4)
>depth 3:   4 Caches (type #4)
> depth 4:  4 Cores (type #5)
>  depth 5: 8 Procs (type #6)