[hwloc-users] NUMA, io and miscellaneous object depths
Hello folks, The function hwloc_get_type_or_above_depth() is supposed to return the depth of objects of type “type" or above. It internally calls hwloc_get_type_depth which returns virtual depths to NUMA, IO and misc objects. In order to retrieve the actual depth of these objects, one needs to call hwloc_get_obj_depth() with virtual depth. Can the documentation be updated to cover this? Or are there plans of changing this behavior? Thanks Kavitha ___ hwloc-users mailing list hwloc-users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/hwloc-users
Re: [hwloc-users] NUMA, io and miscellaneous object depths
Hello I can fix the documentation to say that the function always suceeds and returns the virtual depth for NUMA/IO/Misc. I don't understand your third sentence. If by "actual depth", you mean the depth of a (normal) parent where NUMA are attached (for instance the depth of Package if NUMAs are attached to Packages), see hwloc_get_memory_parents_depth(). However, you may have NUMA/IO/Misc attached to parents at different depths, so it doesn't make much sense in the general case. What do you use this function for? I thought of removing it from 2.0 because it's hard to define a "usual" order for object types (for instance L3 can be above or below NUMA for different modern platforms). Brice Le 14/03/2018 à 20:24, Madhu, Kavitha Tiptur a écrit : > Hello folks, > > The function hwloc_get_type_or_above_depth() is supposed to return the depth > of objects of type “type" or above. It internally calls hwloc_get_type_depth > which returns virtual depths to NUMA, IO and misc objects. In order to > retrieve the actual depth of these objects, one needs to call > hwloc_get_obj_depth() with virtual depth. Can the documentation be updated to > cover this? Or are there plans of changing this behavior? > > Thanks > Kavitha > ___ > hwloc-users mailing list > hwloc-users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/hwloc-users ___ hwloc-users mailing list hwloc-users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/hwloc-users
Re: [hwloc-users] NUMA, io and miscellaneous object depths
Hi This function was used to query depth of hardware objects of a certain type to bind processes to objects at the depth or above in Hydra previously. As you pointed out, the functionality makes no sense with NUMA/IO objects possibly being at different depths or for objects. > On Mar 14, 2018, at 3:00 PM, Brice Goglin wrote: > > Hello > > I can fix the documentation to say that the function always suceeds and > returns the virtual depth for NUMA/IO/Misc. > > I don't understand your third sentence. If by "actual depth", you mean > the depth of a (normal) parent where NUMA are attached (for instance the > depth of Package if NUMAs are attached to Packages), see > hwloc_get_memory_parents_depth(). However, you may have NUMA/IO/Misc > attached to parents at different depths, so it doesn't make much sense > in the general case. > > What do you use this function for? I thought of removing it from 2.0 > because it's hard to define a "usual" order for object types (for > instance L3 can be above or below NUMA for different modern platforms). > > Brice > > > > Le 14/03/2018 à 20:24, Madhu, Kavitha Tiptur a écrit : >> Hello folks, >> >> The function hwloc_get_type_or_above_depth() is supposed to return the depth >> of objects of type “type" or above. It internally calls hwloc_get_type_depth >> which returns virtual depths to NUMA, IO and misc objects. In order to >> retrieve the actual depth of these objects, one needs to call >> hwloc_get_obj_depth() with virtual depth. Can the documentation be updated >> to cover this? Or are there plans of changing this behavior? >> >> Thanks >> Kavitha >> ___ >> hwloc-users mailing list >> hwloc-users@lists.open-mpi.org >> https://lists.open-mpi.org/mailman/listinfo/hwloc-users > > ___ > hwloc-users mailing list > hwloc-users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/hwloc-users ___ hwloc-users mailing list hwloc-users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/hwloc-users
Re: [hwloc-users] NUMA, io and miscellaneous object depths
A follow up question, can the call to hwloc_get_non_io_ancestor_obj() return a numa object? > On Mar 14, 2018, at 3:09 PM, Madhu, Kavitha Tiptur wrote: > > Hi > This function was used to query depth of hardware objects of a certain type > to bind processes to objects at the depth or above in Hydra previously. As > you pointed out, the functionality makes no sense with NUMA/IO objects > possibly being at different depths or for objects. > >> On Mar 14, 2018, at 3:00 PM, Brice Goglin wrote: >> >> Hello >> >> I can fix the documentation to say that the function always suceeds and >> returns the virtual depth for NUMA/IO/Misc. >> >> I don't understand your third sentence. If by "actual depth", you mean >> the depth of a (normal) parent where NUMA are attached (for instance the >> depth of Package if NUMAs are attached to Packages), see >> hwloc_get_memory_parents_depth(). However, you may have NUMA/IO/Misc >> attached to parents at different depths, so it doesn't make much sense >> in the general case. >> >> What do you use this function for? I thought of removing it from 2.0 >> because it's hard to define a "usual" order for object types (for >> instance L3 can be above or below NUMA for different modern platforms). >> >> Brice >> >> >> >> Le 14/03/2018 à 20:24, Madhu, Kavitha Tiptur a écrit : >>> Hello folks, >>> >>> The function hwloc_get_type_or_above_depth() is supposed to return the >>> depth of objects of type “type" or above. It internally calls >>> hwloc_get_type_depth which returns virtual depths to NUMA, IO and misc >>> objects. In order to retrieve the actual depth of these objects, one needs >>> to call hwloc_get_obj_depth() with virtual depth. Can the documentation be >>> updated to cover this? Or are there plans of changing this behavior? >>> >>> Thanks >>> Kavitha >>> ___ >>> hwloc-users mailing list >>> hwloc-users@lists.open-mpi.org >>> https://lists.open-mpi.org/mailman/listinfo/hwloc-users >> >> ___ >> hwloc-users mailing list >> hwloc-users@lists.open-mpi.org >> https://lists.open-mpi.org/mailman/listinfo/hwloc-users > > ___ > hwloc-users mailing list > hwloc-users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/hwloc-users ___ hwloc-users mailing list hwloc-users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/hwloc-users
Re: [hwloc-users] NUMA, io and miscellaneous object depths
Good point. In theory, that's possible because we only look at cpusets (NUMA nodes have cpusets, I/O don't). So the name of the function still matches its behavior. However it won't happen in practice with the current code because I/O are always attached to CPU objects. But it may change in the future with things like processing-in-memory etc. Instead of calling this function, you could do a while (!hwloc_obj_type_is_normal(obj->type)) obj = obj->parent; I'll update the doc too. Thanks. Brice Le 14/03/2018 à 22:16, Madhu, Kavitha Tiptur a écrit : > A follow up question, can the call to hwloc_get_non_io_ancestor_obj() return > a numa object? > >> On Mar 14, 2018, at 3:09 PM, Madhu, Kavitha Tiptur wrote: >> >> Hi >> This function was used to query depth of hardware objects of a certain type >> to bind processes to objects at the depth or above in Hydra previously. As >> you pointed out, the functionality makes no sense with NUMA/IO objects >> possibly being at different depths or for objects. >> >>> On Mar 14, 2018, at 3:00 PM, Brice Goglin wrote: >>> >>> Hello >>> >>> I can fix the documentation to say that the function always suceeds and >>> returns the virtual depth for NUMA/IO/Misc. >>> >>> I don't understand your third sentence. If by "actual depth", you mean >>> the depth of a (normal) parent where NUMA are attached (for instance the >>> depth of Package if NUMAs are attached to Packages), see >>> hwloc_get_memory_parents_depth(). However, you may have NUMA/IO/Misc >>> attached to parents at different depths, so it doesn't make much sense >>> in the general case. >>> >>> What do you use this function for? I thought of removing it from 2.0 >>> because it's hard to define a "usual" order for object types (for >>> instance L3 can be above or below NUMA for different modern platforms). >>> >>> Brice >>> >>> >>> >>> Le 14/03/2018 à 20:24, Madhu, Kavitha Tiptur a écrit : Hello folks, The function hwloc_get_type_or_above_depth() is supposed to return the depth of objects of type “type" or above. It internally calls hwloc_get_type_depth which returns virtual depths to NUMA, IO and misc objects. In order to retrieve the actual depth of these objects, one needs to call hwloc_get_obj_depth() with virtual depth. Can the documentation be updated to cover this? Or are there plans of changing this behavior? Thanks Kavitha ___ hwloc-users mailing list hwloc-users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/hwloc-users >>> ___ >>> hwloc-users mailing list >>> hwloc-users@lists.open-mpi.org >>> https://lists.open-mpi.org/mailman/listinfo/hwloc-users >> ___ >> hwloc-users mailing list >> hwloc-users@lists.open-mpi.org >> https://lists.open-mpi.org/mailman/listinfo/hwloc-users > ___ > hwloc-users mailing list > hwloc-users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/hwloc-users ___ hwloc-users mailing list hwloc-users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/hwloc-users
Re: [hwloc-users] NUMA, io and miscellaneous object depths
Thanks for the response. > On Mar 14, 2018, at 4:28 PM, Brice Goglin wrote: > > Good point. In theory, that's possible because we only look at cpusets > (NUMA nodes have cpusets, I/O don't). So the name of the function still > matches its behavior. > > However it won't happen in practice with the current code because I/O > are always attached to CPU objects. But it may change in the future with > things like processing-in-memory etc. > > Instead of calling this function, you could do a while > (!hwloc_obj_type_is_normal(obj->type)) obj = obj->parent; > > I'll update the doc too. Thanks. > > Brice > > > > Le 14/03/2018 à 22:16, Madhu, Kavitha Tiptur a écrit : >> A follow up question, can the call to hwloc_get_non_io_ancestor_obj() return >> a numa object? >> >>> On Mar 14, 2018, at 3:09 PM, Madhu, Kavitha Tiptur wrote: >>> >>> Hi >>> This function was used to query depth of hardware objects of a certain type >>> to bind processes to objects at the depth or above in Hydra previously. As >>> you pointed out, the functionality makes no sense with NUMA/IO objects >>> possibly being at different depths or for objects. >>> On Mar 14, 2018, at 3:00 PM, Brice Goglin wrote: Hello I can fix the documentation to say that the function always suceeds and returns the virtual depth for NUMA/IO/Misc. I don't understand your third sentence. If by "actual depth", you mean the depth of a (normal) parent where NUMA are attached (for instance the depth of Package if NUMAs are attached to Packages), see hwloc_get_memory_parents_depth(). However, you may have NUMA/IO/Misc attached to parents at different depths, so it doesn't make much sense in the general case. What do you use this function for? I thought of removing it from 2.0 because it's hard to define a "usual" order for object types (for instance L3 can be above or below NUMA for different modern platforms). Brice Le 14/03/2018 à 20:24, Madhu, Kavitha Tiptur a écrit : > Hello folks, > > The function hwloc_get_type_or_above_depth() is supposed to return the > depth of objects of type “type" or above. It internally calls > hwloc_get_type_depth which returns virtual depths to NUMA, IO and misc > objects. In order to retrieve the actual depth of these objects, one > needs to call hwloc_get_obj_depth() with virtual depth. Can the > documentation be updated to cover this? Or are there plans of changing > this behavior? > > Thanks > Kavitha > ___ > hwloc-users mailing list > hwloc-users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/hwloc-users ___ hwloc-users mailing list hwloc-users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/hwloc-users >>> ___ >>> hwloc-users mailing list >>> hwloc-users@lists.open-mpi.org >>> https://lists.open-mpi.org/mailman/listinfo/hwloc-users >> ___ >> hwloc-users mailing list >> hwloc-users@lists.open-mpi.org >> https://lists.open-mpi.org/mailman/listinfo/hwloc-users > > ___ > hwloc-users mailing list > hwloc-users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/hwloc-users ___ hwloc-users mailing list hwloc-users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/hwloc-users