Re: [hwloc-users] Selecting real cores vs HT cores

[Jeff Hammond]

Can someone post docs for this resource halving Squyres claims? I've never
heard of this.

Jeff

On Thursday, December 11, 2014, Samuel Thibault 
wrote:

> Jeff Squyres (jsquyres), le Thu 11 Dec 2014 21:12:27 +, a écrit :
> > When the BIOS is set to enable hyper threading, then several resources
> on the core are split when the machine is booted up (e.g., some of the
> queue depths for various processing units in the core are half the length
> that they are when hyperthreading is disabled in the BIOS).
>
> Perhaps some queues get divided, but most of the resources (such as
> cache, TLB, etc.) are completely available when using only one
> hyperthread, like they would be with HT disabled.
>
> Samuel
> ___
> hwloc-users mailing list
> hwloc-us...@open-mpi.org 
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users
> Link to this post:
> http://www.open-mpi.org/community/lists/hwloc-users/2014/12/1132.php
>


-- 
Jeff Hammond
jeff.scie...@gmail.com
http://jeffhammond.github.io/

Re: [hwloc-users] Selecting real cores vs HT cores

[Samuel Thibault]

Jeff Squyres (jsquyres), le Thu 11 Dec 2014 21:12:27 +, a écrit :
> When the BIOS is set to enable hyper threading, then several resources on the 
> core are split when the machine is booted up (e.g., some of the queue depths 
> for various processing units in the core are half the length that they are 
> when hyperthreading is disabled in the BIOS).

Perhaps some queues get divided, but most of the resources (such as
cache, TLB, etc.) are completely available when using only one
hyperthread, like they would be with HT disabled.

Samuel

Re: [hwloc-users] Selecting real cores vs HT cores

[Jeff Squyres (jsquyres)]

On Dec 11, 2014, at 2:03 PM, Brice Goglin  wrote:

> By the way, if you can't in the BIOS, you may want to disable the
> hyperthread in the kernel:
> 
> for i in $(hwloc-calc --whole-system --po -I pu core:all.pu:0) ; do echo 0 > 
> /sys/devices/system/cpu/cpu$i/online ; done
> 
> (write 1 instead of 0 to reenable them).

But keep in mind that this is the semantic equivalent of using hwloc-bind to 
bind to the first HT in each core.

I.e., disabling HT in the Linux kernel just disables scheduling on the 2nd HT.

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [hwloc-users] Selecting real cores vs HT cores

[Brice Goglin]

Le 11/12/2014 21:51, Brock Palen a écrit :
> When a system has HT enabled is one core presented the real one and one the 
> fake partner?  Or is that not the case?
>
> If wanting to test behavior without messing with the bios how do I select 
> just the 'real cores'  if this is the case?   
>
> I am looking for the equivelent of 
>
> hwloc-bind ALLREALCORES  my.exe
>

Assuming this still makes sense after your discussion with Jeff, you can
easily bind on the first hyperthread of all cores with:

hwloc-bind core:all.pu:0 my.exe


None of the hyperthreads is better than the others afaik.

By the way, if you can't in the BIOS, you may want to disable the
hyperthread in the kernel:

for i in $(hwloc-calc --whole-system --po -I pu core:all.pu:0) ; do echo 0 > 
/sys/devices/system/cpu/cpu$i/online ; done

(write 1 instead of 0 to reenable them).

Brice

Re: [hwloc-users] Selecting real cores vs HT cores

[Brock Palen]

Ok let me expand then.  I don't have control over the bios.

The testing I am doing resides on a cloud provider and from our testing it 
appears that it has HT enabled.  It is ambiguous though to me what I see vs how 
they allocate on their hypervisor. 

I want to see if this has any effect. given the providers advertised CPU types 
they use vs my bare metal ones of the same types everything feels 'half as 
fast'  Thus the question about HT.

Here is the lstopo from the provider:

lstopo-no-graphics

Machine (7484MB)
  Socket L#0 + L3 L#0 (25MB)
L2 L#0 (256KB) + L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0
  PU L#0 (P#0)
  PU L#1 (P#2)
L2 L#1 (256KB) + L1d L#1 (32KB) + L1i L#1 (32KB) + Core L#1
  PU L#2 (P#1)
  PU L#3 (P#3)
  HostBridge L#0
PCI 8086:7010
PCI 1013:00b8
PCI 8086:10ed
  Net L#0 "eth0"


Brock Palen
www.umich.edu/~brockp
CAEN Advanced Computing
XSEDE Campus Champion
bro...@umich.edu
(734)936-1985



> On Dec 11, 2014, at 4:12 PM, Jeff Squyres (jsquyres)  
> wrote:
> 
> I'm not sure you're asking a well-formed question.
> 
> When the BIOS is set to enable hyper threading, then several resources on the 
> core are split when the machine is booted up (e.g., some of the queue depths 
> for various processing units in the core are half the length that they are 
> when hyperthreading is disabled in the BIOS).
> 
> Hence, running a process on a core that only uses a single hyperthread (when 
> HT is enabled) is not quite the same thing as booting up with HT disabled and 
> running that same job on the core.
> 
> Make sense?
> 
> Meaning: if you want to test HT vs. non-HT performance, you really need to 
> change the BIOS settings and reboot, sorry.
> 
> Also, note that if you have HT enabled and you run a single-threaded app 
> bound to a core, it will only use 1 of those HTs -- the other HT will be 
> largely dormant. Meaning: don't expect that running a single-threaded app on 
> a core that has HT enabled will magically take advantage of some performance 
> benefit of aggressive automatic parallelization.  You really need multiple 
> threads in a process to get performance advantages out of HT.
> 
> 
> 
> On Dec 11, 2014, at 12:51 PM, Brock Palen  wrote:
> 
>> When a system has HT enabled is one core presented the real one and one the 
>> fake partner?  Or is that not the case?
>> 
>> If wanting to test behavior without messing with the bios how do I select 
>> just the 'real cores'  if this is the case?   
>> 
>> I am looking for the equivelent of 
>> 
>> hwloc-bind ALLREALCORES  my.exe
>> 
>> Doing some performance study type things.
>> 
>> Thanks,
>> 
>> Brock Palen
>> www.umich.edu/~brockp
>> CAEN Advanced Computing
>> XSEDE Campus Champion
>> bro...@umich.edu
>> (734)936-1985
>> 
>> 
>> 
>> ___
>> hwloc-users mailing list
>> hwloc-us...@open-mpi.org
>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users
>> Link to this post: 
>> http://www.open-mpi.org/community/lists/hwloc-users/2014/12/1126.php
> 
> 
> -- 
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to: 
> http://www.cisco.com/web/about/doing_business/legal/cri/
> 
> ___
> hwloc-users mailing list
> hwloc-us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users
> Link to this post: 
> http://www.open-mpi.org/community/lists/hwloc-users/2014/12/1127.php

Re: [hwloc-users] Selecting real cores vs HT cores

[Jeff Squyres (jsquyres)]

I'm not sure you're asking a well-formed question.

When the BIOS is set to enable hyper threading, then several resources on the 
core are split when the machine is booted up (e.g., some of the queue depths 
for various processing units in the core are half the length that they are when 
hyperthreading is disabled in the BIOS).

Hence, running a process on a core that only uses a single hyperthread (when HT 
is enabled) is not quite the same thing as booting up with HT disabled and 
running that same job on the core.

Make sense?

Meaning: if you want to test HT vs. non-HT performance, you really need to 
change the BIOS settings and reboot, sorry.

Also, note that if you have HT enabled and you run a single-threaded app bound 
to a core, it will only use 1 of those HTs -- the other HT will be largely 
dormant. Meaning: don't expect that running a single-threaded app on a core 
that has HT enabled will magically take advantage of some performance benefit 
of aggressive automatic parallelization.  You really need multiple threads in a 
process to get performance advantages out of HT.



On Dec 11, 2014, at 12:51 PM, Brock Palen  wrote:

> When a system has HT enabled is one core presented the real one and one the 
> fake partner?  Or is that not the case?
> 
> If wanting to test behavior without messing with the bios how do I select 
> just the 'real cores'  if this is the case?   
> 
> I am looking for the equivelent of 
> 
> hwloc-bind ALLREALCORES  my.exe
> 
> Doing some performance study type things.
> 
> Thanks,
> 
> Brock Palen
> www.umich.edu/~brockp
> CAEN Advanced Computing
> XSEDE Campus Champion
> bro...@umich.edu
> (734)936-1985
> 
> 
> 
> ___
> hwloc-users mailing list
> hwloc-us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users
> Link to this post: 
> http://www.open-mpi.org/community/lists/hwloc-users/2014/12/1126.php


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/