Re: [hwloc-devel] upcoming feature removal

[Jeff Squyres (jsquyres)]

On Nov 4, 2014, at 9:10 AM, Brice Goglin  wrote:

>> - pu_children
>> - io_children
>> - misc_children
> 
> OK but what does "pe" stand for? did you mean "pu" to match our PU objects?

I said "pu_children".  Really.  I didn't edit the text above and make it look 
like I didn't typo in the original email.  Really.  Trust me.

:-)

(yes, I meant "pu")

> Unfortunately, children are more than a single field in struct hwloc_obj :)
> 
>  unsigned arity;   /**< \brief Number of children */
>  struct hwloc_obj **children;  /**< \brief Children, \c children[0 .. 
> arity -1] */
>  struct hwloc_obj *first_child;/**< \brief First child */
>  struct hwloc_obj *last_child; /**< \brief Last child */
> 
> I'll review the code to see if we can easily remove first/last_child or so.

My 2 main points:

1. if we're going to have multiple lists, then we might as well call them -- 
and all the related data accompanying them (e.g., first/last_child) what they 
are.  Don't just have "children" and "misc" -- have specific names denoting 
what they are.

2. If we know we're going to have io devices, and we're going to separate them 
out from (pu_)children, then we might as well have a specific list for them.

> So on current x86 machines, we would have this?
> * one level with L1 object with cache attribute "Data"
> * one level with L1 object with cache attribute "Instruction"
> * one level with L2 object with cache attribute "Unified"
> * one level with L3 object with cache attribute "Unified"
> 
> Fortunately, instruction caches are disabled by default (unless somebody
> wants to change that?) so most application will see a single L1 level.

To be honest, I'm not sure what the Right answer is.  It would certainly be 
nice to be able to have a single switch/case to be able to identify all the 
different types of topology items, including the differences between the 
different caches.

Perhaps it would be sufficient to be able to combine the type and the cache 
type together somehow -- e.g., the cache type could be bit-shifted up above the 
type bits, and then you could switch/case on the combined value...?  (waving 
hands furiously...)

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [hwloc-devel] upcoming feature removal

[Brice Goglin]

Le 04/11/2014 14:59, Jeff Squyres (jsquyres) a écrit :
> If you're going to separate them from the normal "children", then how about 
> naming them for what they are?  E.g.:
>
> - pe_children
> - io_children
> - misc_children

OK but what does "pe" stand for? did you mean "pu" to match our PU objects?
Unfortunately, children are more than a single field in struct hwloc_obj :)

  unsigned arity;   /**< \brief Number of children */
  struct hwloc_obj **children;  /**< \brief Children, \c children[0 .. 
arity -1] */
  struct hwloc_obj *first_child;/**< \brief First child */
  struct hwloc_obj *last_child; /**< \brief Last child */

I'll review the code to see if we can easily remove first/last_child or so.


>> * stop having 4 cpusets and 3 nodesets per object and just have 1 cpuset
>> and 1 nodeset depending on topology flags (only allowed, or only online,
>> etc). possibly with ways to switch between modes at runtime
> No, please don't do this.  For big code bases like Open MPI, we don't know 
> how the different consumers of hwloc will use the information.  So we get all 
> of the topo information once, at the beginning of time.  Various different 
> plugins and different parts of OMPI may want different information, and it 
> would kinda defeat the point if they had to re-scan the topo because we 
> didn't initially get the information that they needed.

Ah good point !


>> * stop having a CACHE type + data/instruction/unified + depth, and just
>> have one type for each of them, such as HWLOC_OBJ_CACHE_L1d. the
>> advantage is that you can switch (type) without special-casing the CACHE
>> subtypes. One drawback is that there are many subtypes in existing
>> machines (at least L1[id], L2[idu], L3[idu], L4u).
> Samuel has a good point about "where will it end?".
>
> But I also agree that it's pretty annoying (and Ralph has [rightfully] 
> complained a bunch) to have to special-case the check for the various caches 
> -- it has caused a bunch of code churn in OMPI.
>
> How about making enums for L1 through L5?  That's more than any architecture 
> has today, and gives us a bit of future-proofing.

So on current x86 machines, we would have this?
* one level with L1 object with cache attribute "Data"
* one level with L1 object with cache attribute "Instruction"
* one level with L2 object with cache attribute "Unified"
* one level with L3 object with cache attribute "Unified"

Fortunately, instruction caches are disabled by default (unless somebody
wants to change that?) so most application will see a single L1 level.

And Group remains unchanged (single type value with a depth attribute)
since most people will ignore them anyway as Samuel said.

Brice

Re: [hwloc-devel] upcoming feature removal

[Jeff Squyres (jsquyres)]

On Nov 3, 2014, at 5:49 AM, Brice Goglin  wrote:

> * don't put I/O objects in "normal" children since it confuses programs
> consulting the children list. rather place them under a dedicated child
> pointer special objects such as Misc may go there as well.

If you're going to separate them from the normal "children", then how about 
naming them for what they are?  E.g.:

- pe_children
- io_children
- misc_children

> * stop having 4 cpusets and 3 nodesets per object and just have 1 cpuset
> and 1 nodeset depending on topology flags (only allowed, or only online,
> etc). possibly with ways to switch between modes at runtime

No, please don't do this.  For big code bases like Open MPI, we don't know how 
the different consumers of hwloc will use the information.  So we get all of 
the topo information once, at the beginning of time.  Various different plugins 
and different parts of OMPI may want different information, and it would kinda 
defeat the point if they had to re-scan the topo because we didn't initially 
get the information that they needed.

> * stop having a CACHE type + data/instruction/unified + depth, and just
> have one type for each of them, such as HWLOC_OBJ_CACHE_L1d. the
> advantage is that you can switch (type) without special-casing the CACHE
> subtypes. One drawback is that there are many subtypes in existing
> machines (at least L1[id], L2[idu], L3[idu], L4u).

Samuel has a good point about "where will it end?".

But I also agree that it's pretty annoying (and Ralph has [rightfully] 
complained a bunch) to have to special-case the check for the various caches -- 
it has caused a bunch of code churn in OMPI.

How about making enums for L1 through L5?  That's more than any architecture 
has today, and gives us a bit of future-proofing.

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/