Re: [hwloc-users] memory binding on Knights Landing

[Dave Love]

Brice Goglin  writes:

> Is there anything to fix on the RPM side?

Nothing significant, I think.  The update Fedora version needed slight
adjustment for hwloc-dump-hwdata, at least.

> Intel people are carrefully
> working with RedHat so that hwloc is properly packaged for RHEL. I can
> report bugs if needed.

I can't see a recent hwloc for RHEL (e.g. in RHEL7 beta), but don't get
me started on RHEL and HPC...

I'd originally just rebuilt an updated RHEL6 package, but that had a
load of compatibility stuff in it.  In case it's useful, a clean build
of the updated Fedora package for EPEL7 is under
https://copr.fedorainfracloud.org/coprs/loveshack/livhpc/package/hwloc/

[Off-topic, but there's a little KNL-specific stuff in a sibling repo
(el7-knl), and one or two things with avx512 support in the livhpc one.]

>
>> By the way, is it expected that binding will be slow on it?  hwloc-bind
>> is ~10 times slower (~1s) than on two-socket sandybridge, and ~3 times
>> slower than on a 128-core, 16-socket system.
>
> Binding itself shouldn't be slower. But hwloc's topology discovery
> (which is performed by hwloc-bind before actual binding) is slower on
> KNL than on "normal" nodes. The overhead is basically linear with the
> number of hyperthreads, and KNL sequential perf is lower than your other
> nodes.

Right, thanks.  I shouldn't have confused them, especially as I was
looking at things concerned with discovery...  I don't remember how that
works, if I ever did in detail, but I was expecting it to scale with the
number of kernel file opens.

>
> The easy fix is to export the topology to XML with lstopo foo.xml and
> then tell all hwloc users to load from XML:

Yes; I think I even noted that somewhere in the SGE source...

By the way, in case this attracts the attention of anyone who might do
what I did:  hwloc-distances produces no output, as a feature
.
___
hwloc-users mailing list
hwloc-users@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/hwloc-users

Re: [hwloc-users] memory binding on Knights Landing

[Dave Love]

Jeff Hammond  writes:

>> By the way, is it expected that binding will be slow on it?  hwloc-bind
>> is ~10 times slower (~1s) than on two-socket sandybridge, and ~3 times
>> slower than on a 128-core, 16-socket system.
>>
>> Is this a bottleneck in any application?  Are there codes bindings memory
> frequently?

As Brice pointed out, I was stupidly confusing discovery and binding,
but there are cases where a second to do the discovery could be
significant, even if I might not consider them too sensible.

> Because most things inside the kernel are limited by single-threaded
> performance, it is reasonable for them to be slower than on a Xeon
> processor, but I've not seen slowdowns that high.
>
> Jeff

Yes.  Actually I was originally comparing with a slower 64-core
Interlagos, but that was with an older hwloc.
___
hwloc-users mailing list
hwloc-users@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/hwloc-users