Hello,

We currently have a single object type for caches (HWLOC_OBJ_CACHE) but
all machines have multiple levels of caches (usually, L1i, L1d, L2 and
L3). Functions like hwloc_get_obj_by_type() don't work for caches
because of these multiple levels. We got multiple complaints about this
because this requires special cases like:
https://github.com/open-mpi/ompi/blob/master/orte/mca/rmaps/base/rmaps_base_binding.c#L910

The new "cachetypes" branch splits the single HWLOC_OBJ_CACHE into
multiple types, one per level:
* HWLOC_OBJ_L1CACHE, ..., HWLOC_OBJ_L5CACHE (those are either Data or
Unified caches, and we have never seen anything above L4).
* HWLOC_OBJ_L1ICACHE, ..., HWLOC_OBJ_L3ICACHE (modern processors have a
L1i, and I have never seen anything above L2i)

Random notes:

* Your switch cases on object types will need 8 lines to match all cache
types:
https://github.com/open-mpi/hwloc/blob/cachetypes/hwloc/topology-xml.c#L173
Ranges in switch cases are not standard C yet unfortunately.

* There is a hwloc_obj_type_is_cache(type) for testing if a given type
is a cache (and variants for data and instruction caches).
https://github.com/open-mpi/hwloc/blob/cachetypes/include/hwloc/helper.h#L453

* If you want to ignore caches entirely in your topology, you need to
call the ignore function 8 times :/ I am planning to rework the ignoring
API in v2.0, we'll see if we can make that easier.

* Contrary to the renaming of SOCKET into PACKAGE or NODE into NUMANODE,
we can't #define the old HWLOC_OBJ_CACHE to a new type to ease
forward-compatibility. I think it's better to force users to update
their code anyway. In the vast majority of cases, the new code will
be shorter thanks to no cache-specific hacks anymore.

* There will be XML backward-compatiblity: old XML will be converted to
new cache types.

* You may still have multiple levels of the same type in case of "Groups
of Groups" (usually on large SGI UV machines). That's why
HWLOC_TYPE_DEPTH_MULTIPLE will still exist unfortunately. Fortunately,
you probably never lookup "Group" objects "by_type".

If you have comments, let me know.

Brice

Reply via email to