Re: [hwloc-users] BGQ question.

2014-03-30 Thread Christopher Samuel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi John, On 28/03/14 21:04, Biddiscombe, John A. wrote: > Just as a follow up to this thread. I spoke with someone from IBM > and they tell me that 2 cores of 4 hardware threads each are hidden > from the kernel (how do they do that?) and used for

Re: [hwloc-users] BGQ question.

2014-03-28 Thread Biddiscombe, John A.
> -Original Message- > From: hwloc-users [mailto:hwloc-users-boun...@open-mpi.org] On Behalf > Of Chris Samuel > Sent: 26 March 2014 13:42 > To: Hardware locality user list > Subject: Re: [hwloc-users] BGQ question. > > On Wed, 26 Mar 2014 11:56:08 AM Biddiscombe, John A. wr

Re: [hwloc-users] BGQ question.

2014-03-26 Thread Chris Samuel
On Wed, 26 Mar 2014 06:49:30 AM Kenneth A. Lloyd wrote: > Is there a difference between INK and CNK? BG/Q IO nodes run RHEL6 with a patched distro kernel. BGQ compute nodes run CNK which is a custom kernel written in C++ and licensed under the Eclipse Public License (EPL). It does not

Re: [hwloc-users] BGQ question.

2014-03-26 Thread Kenneth A. Lloyd
Subject: Re: [hwloc-users] BGQ question. On Wed, 26 Mar 2014 11:56:08 AM Biddiscombe, John A. wrote: > I can’t test this as the system is down for maintenance, but if memory > serves me correctly, the GCC compiled lstopo also showed 60 cores > instead of 64/68. It can only re

Re: [hwloc-users] BGQ question.

2014-03-26 Thread Chris Samuel
On Wed, 26 Mar 2014 11:56:08 AM Biddiscombe, John A. wrote: > I can’t test this as the system is down for maintenance, but if memory > serves me correctly, the GCC compiled lstopo also showed 60 cores instead > of 64/68. It can only report what the kernel reports and it appears your kernel is

Re: [hwloc-users] BGQ question.

2014-03-26 Thread Brice Goglin
Le 26/03/2014 01:00, Christopher Samuel a écrit : > On 26/03/14 01:34, Biddiscombe, John A. wrote: > > > If I compile on the login node, but run lstopo on the ION, I get > > this (wrong, below) > > If you build this with GCC (the standard system one, not the > cross-compiler for BGQ) does it still

Re: [hwloc-users] BGQ question.

2014-03-25 Thread Biddiscombe, John A.
Chris > Out of interest, why on an I/O node? I'm targeting the BGQ BGAS nodes with flash cards installed. We've done tests with GPFS mounted on the flash and are trying to get comparable results with an in-house driver. JB

Re: [hwloc-users] BGQ question.

2014-03-25 Thread Chris Samuel
On Tue, 25 Mar 2014 08:22:51 AM Jeff Hammond wrote: > I am inclined to think the issue us that ion Linux uses only 16 cores. When > hwloc does n-- for CNK to skip the 17th (OS) core, it gets the wrong answer > for Linux. Not here, it exposes all 17 cores on our BG/Q IO nodes. I wonder if it's

Re: [hwloc-users] BGQ question.

2014-03-25 Thread Jeff Hammond
I am inclined to think the issue us that ion Linux uses only 16 cores. When hwloc does n-- for CNK to skip the 17th (OS) core, it gets the wrong answer for Linux. Just check for Linux support and use /proc/cpuinfo and don't adjust manually. I'm not sure hwloc on BGQ ion ion needs and special

Re: [hwloc-users] BGQ question.

2014-03-25 Thread Jeff Hammond
es are open for some special projects where we are trying > to customise the IO. > > JB > > From: Brice Goglin [mailto:brice.gog...@inria.fr] > Sent: 25 March 2014 08:43 > To: Hardware locality user list; Biddiscombe, John A. > Subject: Re: [hwloc-users] BGQ question. >

Re: [hwloc-users] BGQ question.

2014-03-25 Thread Samuel Thibault
Biddiscombe, John A., le Tue 25 Mar 2014 08:56:02 +, a écrit : > Looking at /proc/cpuinfo on the io node itself, I see only 60 cores listed. I > wonder if they’ve reserved one socket of 4 cores for IO purposes That's possible, yes. > and in fact hwloc is seeing the correct information. At

Re: [hwloc-users] BGQ question.

2014-03-25 Thread Biddiscombe, John A.
Goglin [mailto:brice.gog...@inria.fr] Sent: 25 March 2014 09:28 To: Biddiscombe, John A.; Hardware locality user list Subject: Re: [hwloc-users] BGQ question. Can you run hwloc-gather-topology foo and send the resulting foo.tar.bz2 ? If the tarball is too bug, feel free to send it to me in a private

Re: [hwloc-users] BGQ question.

2014-03-25 Thread Brice Goglin
ere we > are trying to customise the IO. > > > > JB > > > > *From:*Brice Goglin [mailto:brice.gog...@inria.fr] > *Sent:* 25 March 2014 08:43 > *To:* Hardware locality user list; Biddiscombe, John A. > *Subject:* Re: [hwloc-users] BGQ question. >

Re: [hwloc-users] BGQ question.

2014-03-25 Thread Biddiscombe, John A.
for some special projects where we are trying to customise the IO. JB From: Brice Goglin [mailto:brice.gog...@inria.fr] Sent: 25 March 2014 08:43 To: Hardware locality user list; Biddiscombe, John A. Subject: Re: [hwloc-users] BGQ question. Wait, I missed the "io node" part of your first mai

Re: [hwloc-users] BGQ question.

2014-03-25 Thread Brice Goglin
./configure >--prefix=/gpfs/bbp.cscs.ch/home/biddisco/apps/clang/hwloc-1.8.1 > >should I rerun with something set? > >Thanks > >JB > > >From: hwloc-users [mailto:hwloc-users-boun...@open-mpi.org] On Behalf >Of Brice Goglin >Sent: 25 March 2014 08:04 >To:

Re: [hwloc-users] BGQ question.

2014-03-25 Thread Biddiscombe, John A.
with something set? Thanks JB From: hwloc-users [mailto:hwloc-users-boun...@open-mpi.org] On Behalf Of Brice Goglin Sent: 25 March 2014 08:04 To: Hardware locality user list Subject: Re: [hwloc-users] BGQ question. Le 25/03/2014 07:51, Biddiscombe, John A. a écrit : I'm compiling hwloc using

Re: [hwloc-users] BGQ question.

2014-03-25 Thread Brice Goglin
Le 25/03/2014 07:51, Biddiscombe, John A. a écrit : > > I'm compiling hwloc using clang (bgclang++11 from ANL) to run on IO > nodes af a BGQ. It seems to have compiled ok, and when I run lstopo, I > get an output like this (below), which looks reasonable, but there are > 15 sockets instead of 16.

[hwloc-users] BGQ question.

2014-03-25 Thread Biddiscombe, John A.
I'm compiling hwloc using clang (bgclang++11 from ANL) to run on IO nodes af a BGQ. It seems to have compiled ok, and when I run lstopo, I get an output like this (below), which looks reasonable, but there are 15 sockets instead of 16. I'm a little worried because the first time I compiled, I