Re: [hwloc-users] BGQ question.

2014-03-25 Thread Brice Goglin
ere we > are trying to customise the IO. > > > > JB > > > > *From:*Brice Goglin [mailto:brice.gog...@inria.fr] > *Sent:* 25 March 2014 08:43 > *To:* Hardware locality user list; Biddiscombe, John A. > *Subject:* Re: [hwloc-users] BGQ question. >

Re: [hwloc-users] BGQ question.

2014-03-25 Thread Brice Goglin
./configure >--prefix=/gpfs/bbp.cscs.ch/home/biddisco/apps/clang/hwloc-1.8.1 > >should I rerun with something set? > >Thanks > >JB > > >From: hwloc-users [mailto:hwloc-users-boun...@open-mpi.org] On Behalf >Of Brice Goglin >Sent: 25 March 2014 08:04 >To:

Re: [hwloc-users] BGQ question.

2014-03-25 Thread Brice Goglin
Le 25/03/2014 07:51, Biddiscombe, John A. a écrit : > > I'm compiling hwloc using clang (bgclang++11 from ANL) to run on IO > nodes af a BGQ. It seems to have compiled ok, and when I run lstopo, I > get an output like this (below), which looks reasonable, but there are > 15 sockets instead of 16.

Re: [OMPI users] hwloc error in topology.c in OMPI 1.6.5

2014-03-03 Thread Brice Goglin
Le 03/03/2014 23:02, Gus Correa a écrit : > I rebooted the node and ran hwloc-gather-topology again. > This turn it didn't throw any errors on the terminal window, > which may be a good sign. > > [root@node14 ~]# hwloc-gather-topology /tmp/`date > +"%Y%m%d%H%M"`.$(uname -n) > Hierarchy gathered in

Re: [OMPI users] Heterogeneous cluster problem - mixing AMD and Intel nodes

2014-03-02 Thread Brice Goglin
What's your mpirun or mpiexec command-line? The error "BTLs attempted: self sm tcp" says that it didn't even try the MX BTL (for Open-MX). Did you use the MX MTL instead? Are you sure that you actually use Open-MX when not mixing AMD and Intel nodes? Brice Le 02/03/2014 08:06, Victor a écrit :

Re: [OMPI users] hwloc error in topology.c in OMPI 1.6.5

2014-02-28 Thread Brice Goglin
Le 28/02/2014 02:48, Ralph Castain a écrit : > Remember, hwloc doesn't actually "sense" hardware - it just parses files in > the /proc area. So if something is garbled in those files, hwloc will report > errors. Doesn't mean anything is wrong with the hardware at all. For the record, that's not

Re: [OMPI users] hwloc error in topology.c in OMPI 1.6.5

2014-02-28 Thread Brice Goglin
Hello Gus, I'll need the tarball generated by gather-topology on node14 to debug this. node15 doesn't have any issue. We've seen issues on AMD machines because of buggy BIOS reporting incompatible Socket and NUMA info. If node14 doesn't have the same BIOS version as other nodes, that could explain

Re: [hwloc-users] [hwloc-announce] Hardware Locality (hwloc) v1.8.1 released

2014-02-13 Thread Brice Goglin
Le 13/02/2014 22:25, Jiri Hladky a écrit : > Hi Brice, > > when compiling hwloc-1.8.1 I have seen these warnings. Could you > please check them? fread() warnings come from fread() on kernel sysfs files, so it's very unlikely that we read totally buggy data from there. One day we'll fix this,

Re: [OMPI users] "bind-to l3chace" with r30643 in ticket #4240 dosen't work

2014-02-12 Thread Brice Goglin
Is there anything we could do in hwloc to improve this? (I don't even know the exact piece of code you are refering to) Brice Le 12/02/2014 02:46, Ralph Castain a écrit : > Okay, I fixed it. Keep getting caught by a very, very unfortunate design flaw > in hwloc that forces you to treat cache's

Re: [hwloc-users] Using hwloc to map GPU layout on system

2014-02-07 Thread Brice Goglin
quot;nvml2" > GPU L#5 "nvml3" > GPU L#7 "nvml0" > GPU L#9 "nvml1" > > Is the L# always going to be in the oder I would expect? Because then I > already have my map then. Brice > > Brock Palen >

Re: [hwloc-users] Using hwloc to map GPU layout on system

2014-02-06 Thread Brice Goglin
lowing the PCI bus order? We may want to talk to NVIDIA to get a clarification about all this. Brice > > Brock Palen > www.umich.edu/~brockp > CAEN Advanced Computing > XSEDE Campus Champion > bro...@umich.edu > (734)936-1985 > > > > On Feb 5, 2014, at 1:19 AM, Brice

Re: [hwloc-users] Using hwloc to map GPU layout on system

2014-02-05 Thread Brice Goglin
Hello Brock, Some people reported the same issue in the past and that's why we added the "nvml" objects. CUDA reorders devices by "performance". Batch-schedulers are somehow supposed to use "nvml" for managing GPUs without actually using them with CUDA directly. And the "nvml" order is the

Re: [hwloc-users] misleading cache size on AMD Opteron 6348?

2014-01-31 Thread Brice Goglin
Hello, Your BIOS reports invalid L3 cache information. On these processors, the L3 is shared by 6 cores, it covers 6 cores of an entire half-socket NUMA node. But the BIOS says that some L3 are shared between 4 cores, others by 6 cores. And worse it says that some L3 is shared by some cores from

Re: [hwloc-users] Having trouble getting CPU Model string on Windows 7 x64

2014-01-29 Thread Brice Goglin
en-mpi.org/community/lists/hwloc-devel/2014/01/4043.php Le 29/01/2014 06:50, Robin Scher a écrit : > Hi Brice > > This works great now. Thank you for your help! > -robin > > Robin Scher > ro...@uberware.net > +1 (213) 448-0443 > > > > On Jan 28, 2014, at 7:47

Re: [hwloc-users] Finding closest host bridge

2014-01-28 Thread Brice Goglin
The bridge cannot be "not connected to anything". All objects have a parent (and are a child of that parent) except the very-top root object. Theoretically, the bridge could be connected anywhere. In practice it's connected to a NUMA node, a root object, or (rarely) a group of numa nodes. The

Re: [hwloc-users] CPU info on ARM

2014-01-28 Thread Brice Goglin
; models executing in the same SMP system)." >> >> He passed the question on to another ARM guy, asking for further detail. >> I'll pass on what he says. >> >> >> >> On Jan 28, 2014, at 3:39 AM, Brice Goglin <brice.gog...@inria.fr>

Re: [hwloc-users] Having trouble getting CPU Model string on Windows 7 x64

2014-01-28 Thread Brice Goglin
Le 28/01/2014 14:31, Brice Goglin a écrit : > Le 28/01/2014 13:00, Samuel Thibault a écrit : >> Brice Goglin, le Tue 28 Jan 2014 12:46:24 +0100, a écrit : >>> 42: xchg %ebx,%rbx >>> >>> I guess having both ebx and rbx on these lines isn't OK. On Linux, I ge

Re: [hwloc-users] Having trouble getting CPU Model string on Windows 7 x64

2014-01-28 Thread Brice Goglin
Le 28/01/2014 13:00, Samuel Thibault a écrit : > Brice Goglin, le Tue 28 Jan 2014 12:46:24 +0100, a écrit : >> 42: xchg %ebx,%rbx >> >> I guess having both ebx and rbx on these lines isn't OK. On Linux, I get >> rsi instead of ebx, no problem. >> >> Samuel

Re: [hwloc-users] Having trouble getting CPU Model string on Windows 7 x64

2014-01-28 Thread Brice Goglin
Le 28/01/2014 09:57, Brice Goglin a écrit : > I will debug a bit more to see if it's actually a 64bit cpuid problem > on windows. The x86 backend is entirely disabled in the 64bit windows build because configure fails to compile the cpuid assembly (in my mingw64 with gcc 4.7). It says

Re: [hwloc-users] Having trouble getting CPU Model string on Windows 7 x64

2014-01-28 Thread Brice Goglin
Le 28/01/2014 09:46, Robin Scher a écrit : > Hi, thanks for responding. > > The CPUModel is definitely available on this machine. A 32 bit process > on the same machine correctly finds the model name using code that > calls the cpuid inline assembly to get it, and the machine itself is a > VM

[hwloc-users] CPU info on ARM

2014-01-28 Thread Brice Goglin
Hello, Is anybody familiar with ARM CPUs? I am adding more CPU information because Intel needs more: CPUVendor=GenuineIntel CPUModel=Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHz CPUModelNumber=45 CPUFamilyNumber=6 Would something similar be useful for ARM? What are the fields below from

Re: [hwloc-users] How to build hwloc static to link into a shared lib on Linux

2014-01-18 Thread Brice Goglin
Maybe try to disable some dependencies such as pci in hwloc (--disable-pci), I wouldn't be surprised if there were issues there. If that helps, please let us know what was enabled before (libpciaccess (default), or libpci/pciutils (--enable-libpci)). Brice Le 18/01/2014 07:23, Robin Scher a

Re: [hwloc-users] hwloc errors on program startup

2014-01-17 Thread Brice Goglin
Hello, Linux says socket 0 contains processors 0-7 and socket 1 contains 8-15, while NUMA node 0 contains processors 0-3+8-11 and NUMA node 1 contains processors 4-7+12-15. Given why I read about Opteron 6320 online, the problem is that NUMA 0 should be replaced with two NUMA nodes with

Re: [hwloc-users] hwloc problem on SGI machine

2014-01-11 Thread Brice Goglin
Le 11/01/2014 01:58, Chris Samuel a écrit : > On Sat, 11 Jan 2014 11:54:17 AM Chris Samuel wrote: > >> We've got both an older Altix XE cluster and a UV10 (both running RHEL) I >> can test on if it's useful? > Forgot I already had both 1.7.2 and 1.8 built for both - all fine (RHEL6.4). > This

Re: [hwloc-users] hwloc problem on SGI machine

2014-01-10 Thread Brice Goglin
Le 11/01/2014 00:27, Jeff Squyres (jsquyres) a écrit : > Jeff Becker (CC'ed) reported to me a failure with hwloc 1.7.2 (in OMPI > trunk). I had him verify this with a standalone hwloc 1.7.2, and then had > him try standalone hwloc 1.8 as well -- all got the same failure. > > Here's what he's

Re: [hwloc-users] [windows] build from source using visual studio

2014-01-08 Thread Brice Goglin
anks, > > Eloi > > > > *From:*hwloc-users [mailto:hwloc-users-boun...@open-mpi.org] *On > Behalf Of *Brice Goglin > *Sent:* mercredi 8 janvier 2014 11:49 > *To:* hwloc-us...@open-mpi.org > *Subject:* Re: [hwloc-users] [windows] build from source using visual > studio

Re: [hwloc-users] [windows] build from source using visual studio

2014-01-08 Thread Brice Goglin
dress(0x07FF7E1B > [c:\windows\system32\PSAPI.DLL], "QueryWorkingSetEx") called from " > XXX\bin\LIBHWLOC-5.DLL" at address 0x69E9419E and returned > 0x07FF7E1B2E60 by thread 1. > > 00:00:00.625: First chance exception 0xC094 (Integer Div

Re: [hwloc-users] [windows] hwloc_get_proc_cpubind issue, even with current process handle as 2nd parameter

2014-01-06 Thread Brice Goglin
Hello, On windows, set/get_cpubind() are implemented by called set/get_proc_cpubind() with GetCurrentProcess() as a 2nd argument, so your code should be doing exactly the same, and I don't see what could be going on here. Do you pass any flag in these cpubind functions ? Do you get any useful

Re: [OMPI users] "-bind-to numa" of openmpi-1.7.4rc1 dosen't work for our magny cours based 32 core node

2013-12-20 Thread Brice Goglin
I don't think there's any such difference. Also, all these NUMA architectures are reported the same by hwloc, and therefore used the same in Open MPI. And yes, L3 and NUMA are topologically-identical on AMD Magny-Cours (and most recent AMD and Intel platforms). Brice Le 20/12/2013 11:33,

Re: [OMPI users] slowdown with infiniband and latest CentOS kernel

2013-12-18 Thread Brice Goglin
hwloc-ps (and lstopo --top) are better at showing process binding but they lack a nice pseudographical interface with dynamic refresh. htop uses hwloc internally iirc, so there's hope we'll have everything needed in htop one day ;) Brice Dave Love a écrit : >John

Re: [hwloc-users] windows PCI locality (was; DELL 8 core machine + Quadro K5000 GPU Card...)

2013-11-19 Thread Brice Goglin
data[0] = 0; > > DEVPROPTYPE type; > > DEVPROPKEY key = DEVPKEY_Numa_Proximity_Domain; > > > > lastError = 0; > > > > ret = SetupDiGetDeviceProperty(hNvDevInfo, > , , , (PBYTE)[0], 20*sizeof(int), NULL

Re: [hwloc-users] Regarding the Dell 8 core machine with GPUs

2013-11-18 Thread Brice Goglin
't contain the kernel version ("uname -a" would be more useful) but I don't need this information anymore anyway. Looks like I am ready to release the final hwloc v1.8 now :) Brice Le 18/11/2013 04:17, Solibakke Per Bjarte a écrit : > Dear Brice Goglin > > Sorry, there mus

Re: [hwloc-users] windows PCI locality (was; DELL 8 core machine + Quadro K5000 GPU Card...)

2013-11-18 Thread Brice Goglin
This seems unrelated since he seems to be running Linux anyway. We got that information a while ago but I couldn't do anything with it because (I think) I didn't have access to a Windows release that supported this. And, bigger problem, I don't have access to a Windows machine with more than one

Re: [hwloc-users] DELL 8 core machine + Quadro K5000 GPU Card...

2013-11-18 Thread Brice Goglin
Le 18/11/2013 02:14, Solibakke Per Bjarte a écrit : > Hello > > I recently got access to a very interesting and powerful machine: Dell > 8 core + GPU Quadro K5000 (96 cores). > A total of 1536 cores in the original machine configuration. Hello GPU cores are not real cores so I am not sure your

Re: [hwloc-users] [hwloc-announce] Hardware locality (hwloc) v1.8rc1 released

2013-11-09 Thread Brice Goglin
index1 object:index2 is easy to write, I'd vote for not making the code too complex. Brice > > Thanks a lot! > Jirka > > > On Wed, Nov 6, 2013 at 3:06 PM, Brice Goglin <brice.gog...@inria.fr > <mailto:brice.gog...@inria.fr>> wrote: > > The Hardware Locality

Re: [hwloc-users] [WARNING: A/V UNSCANNABLE]Re: [OMPI users] SIGSEGV in opal_hwlock152_hwlock_bitmap_or.A // Bug in 'hwlock" ?

2013-11-04 Thread Brice Goglin
losed". Brice Le 04/11/2013 22:33, Paul Kapinos a écrit : > Hello again, > I'm not allowed to publish to Hardware locality user list so I omit it > now. > > On 11/04/13 14:19, Brice Goglin wrote: >> Le 04/11/2013 11:44, Paul Kapinos a écrit : >>> Hello all, >>&

Re: [hwloc-users] [OMPI users] SIGSEGV in opal_hwlock152_hwlock_bitmap_or.A // Bug in 'hwlock" ?

2013-11-04 Thread Brice Goglin
Brice > > Many thanks for your help! > Best > > Paul > > pk224850@linuxitvc00:~/SVN/mpifasttest/trunk[511]lstopo 1.5 > $ lstopo lstopo_linuxitvc00_1.5.txt > $ lstopo lstopo_linuxitvc00_1.5.xml > > > > > > On 11/01/13 15:37, Brice Goglin wrote: >> Sor

Re: [hwloc-users] [OMPI users] SIGSEGV in opal_hwlock152_hwlock_bitmap_or.A // Bug in 'hwlock" ?

2013-11-01 Thread Brice Goglin
Sorry, I missed the mail on OMPI-users. This hwloc looks vry old. We don't have Misc objects instead of Groups since we switched from 0.9 to 1.0. You should regenerate the XML file with a hwloc version that came out after the big bang (or better, after the asteroid killed the

Re: [hwloc-users] meaning of "physical CPU" for get_last_cpu_location()

2013-10-05 Thread Brice Goglin
Hello, The doc says "get the last physical CPU where the current process or thread ran". As usual, there's no well-defined portable way to identify those things. On AMD, it's a core. On Intel, it's a hyperthread, and could also be a core if HT is disabled. On other archs/systems, it may be called

Re: [hwloc-users] CPU binding

2013-10-03 Thread Brice Goglin
Le 03/10/2013 02:56, Panos Labropoulos a écrit : > Hallo, > > > I initially posted this at us...@open-mpi.org . > > We seem to be unable to to set the cpu binding on a cluster consisting > of Dell M420/M610 systems: > > [jallan@hpc21 ~]$ cat report-bindings.sh #!/bin/sh

Re: [hwloc-users] [hwloc-announce] Hardware locality (hwloc) v1.7.2rc1 released

2013-08-29 Thread Brice Goglin
put in 1.7.2 ? (see also my other email I sent to > you 2 minutes ago). > > Jirka > > > > > On Thu, Aug 29, 2013 at 11:32 AM, Brice Goglin <brice.gog...@inria.fr > <mailto:brice.gog...@inria.fr>> wrote: > > The Hardware Locality (hwloc) te

Re: [OMPI users] Mixing Linux's CPU-shielding with mpirun's bind-to-core

2013-08-18 Thread Brice Goglin
Le 18/08/2013 14:51, Siddhartha Jana a écrit : > > If all the above works and does not return errors (you should > check that > your application's PID is in /dev/cpuset/socket0/tasks while running), > bind-to-core won't clash with it, at least when using a OMPI that uses >

Re: [OMPI users] Mixing Linux's CPU-shielding with mpirun's bind-to-core

2013-08-18 Thread Brice Goglin
Le 18/08/2013 05:34, Siddhartha Jana a écrit : > Hi, > > My requirement: > 1. Avoid the OS from scheduling tasks on cores 0-7 allocated to my > process. > 2. Avoid rescheduling of processes to other cores. > > My solution: I use Linux's CPU-shielding. > [ Man page: >

Re: [OMPI users] knem/openmpi performance?

2013-07-18 Thread Brice Goglin
Le 18/07/2013 13:23, Dave Love a écrit : > Mark Dixon writes: > >> On Mon, 15 Jul 2013, Elken, Tom wrote: >> ... >>> Hope these anecdotes are relevant to Open MPI users considering knem. >> ... >> >> Brilliantly useful, thanks! It certainly looks like it may be greatly >>

Re: [OMPI users] Question on handling of memory for communications

2013-07-08 Thread Brice Goglin
. > > Anyways, as long as the memory performance difference is a the levels > you mentioned then there is no "big" issue. Most likely the device > driver get space from the same numa domain that of the socket the HCA > is attached to. > > Thanks for trying it out &

Re: [OMPI users] Question on handling of memory for communications

2013-07-08 Thread Brice Goglin
On a dual E5 2650 machine with FDR cards, I see the IMB Pingpong throughput drop from 6000 to 5700MB/s when the memory isn't allocated on the right socket (and latency increases from 0.8 to 1.4us). Of course that's pingpong only, things will be worse on a memory-overloaded machine. But I don't

Re: [hwloc-users] Open-mpi + hwloc ...

2013-06-21 Thread Brice Goglin
Hello, hwloc can only tell where CPU/device are, and place programs on the right CPUs. hwloc isn't going to convert your parallel program into a GPU program. If you want to use NVIDIA GPUs, you have to rewrite your program using CUDA, OpenCL, or a high-level heterogeneous langage. Brice Le

Re: [hwloc-users] hwloc on Xeon Phi

2013-06-18 Thread Brice Goglin
Le 18/06/2013 08:52, pinak panigrahi a écrit : > Hi, how do I use hwloc on Intel Xeon Phi. I have written codes that > use it for Sandybridge. Hello, If you really mean 'inside the Xeon Phi", it should just work and report all available Phi cores. If you mean managing the Phi internal topology

Re: [hwloc-users] Windows binaries miss lib file

2013-05-20 Thread Brice Goglin
now. All earlier releases (except v0.9) were already OK. Final v1.7.1 expected today or wednesday. Brice Le 20/05/2013 18:45, Brice Goglin a écrit : > Thanks, there was indeed an issue on the machine that builds the Windows > zipballs. I am fixing this. Should be fixed in 1.7.1. If a

Re: [hwloc-users] Windows binaries miss lib file

2013-05-20 Thread Brice Goglin
Thanks, there was indeed an issue on the machine that builds the Windows zipballs. I am fixing this. Should be fixed in 1.7.1. If anybody needs updated earlier Windows zipballs, please let me know. Brice Le 20/05/2013 14:19, Hartmut Kaiser a écrit : > Hey all, > > The V1.7 (and V1.7.1-rc1)

Re: [hwloc-users] [EXTERNAL] Re: Many queries creating slow performance

2013-03-06 Thread Brice Goglin
H i.e. Hydra, then I'm sure Intel will be > committed to helping fix it since Intel MPI is using Hydra as the > launcher on systems like Stampede. > > Best, > > Jeff > > On Tue, Mar 5, 2013 at 3:05 PM, Brice Goglin <brice.gog...@inria.fr> > wrote: > > Just te

Re: [hwloc-users] Many queries creating slow performance

2013-03-05 Thread Brice Goglin
Just tested on a 96-core shared-memory machine. Running OpenMPI 1.6 mpiexec lstopo, here's the execution time (mpiexec launch time is 0.2-0.4s) 1 rank : 0.2s 8 ranks: 0.3-0.5s depending on binding (packed or scatter) 24ranks: 0.8-3.7s depending on binding 48ranks: 2.8-8.0s depending on

Re: [hwloc-users] Trying to get last cpu location

2013-03-05 Thread Brice Goglin
unning on cpuset %s\n", str); > free(str); > > hwloc_cpuset_from_nodeset(topology, cpuset, nodeset); > if( hwloc_set_membind_nodeset(topology, nodeset, HWLOC_MEMBIND_BIND, > HWLOC_MEMBIND_THREAD) < 0 ) > abort(); > > ... > > I omitted som

Re: [hwloc-users] Many queries creating slow performance

2013-03-05 Thread Brice Goglin
Hello Simon, I don't think anybody every benchmarked this, but people have been complaining this problem appearing on large machines at some point. I have a large SGI machine at work, I'll see if I can reproduce this. One solution is to export the topology to XML once and then have all your MPI

Re: [hwloc-users] Zero cache line size on Power7?

2013-03-04 Thread Brice Goglin
gt; No tarball is generated. > > -erik > > > > On Wed, Feb 27, 2013 at 5:19 PM, Brice Goglin <brice.gog...@inria.fr > <mailto:brice.gog...@inria.fr>> wrote: > > I think I've seen cases where the device-tree contains 0 for such > line sizes. >

Re: [hwloc-users] Zero cache line size on Power7?

2013-02-27 Thread Brice Goglin
I think I've seen cases where the device-tree contains 0 for such line sizes. I guess we should document that the line size is 0 means unknown. Can you send the tarball generated by hwloc-gather-topology ? (send it only to, in a private email) Brice Le 27/02/2013 23:11, Erik Schnetter a écrit

Re: [OMPI users] Core ids not coming properly

2013-02-15 Thread Brice Goglin
IntelMPI binds processes by default, while OMPI doesn't. What's your mpiexec/mpirun command-line? Brice Le 15/02/2013 17:34, Kranthi Kumar a écrit : > Hello Sir > > Here below is the code which I wrote using hwloc for getting the > bindings of the processes. > I tested this code on SDSC Gordon

Re: [hwloc-users] hwloc on Blue Gene/Q?

2013-02-11 Thread Brice Goglin
nal 6 (SIGABRT) in a routine > called ".raise". > > I'd be happy to help debug this. How? > > -erik > > > > > On Sat, Feb 9, 2013 at 5:46 PM, Brice Goglin <brice.gog...@inria.fr > <mailto:brice.gog...@inria.fr>> wrote: > > The new "bg

Re: [OMPI users] how to find the binding of each rank on the local machine

2013-02-10 Thread Brice Goglin
Le 10/02/2013 22:47, Ralph Castain a écrit : > I honestly have no idea what you mean. Are you talking about inside an MPI > application? Do you mean from inside the MPI layer? Inside ORTE? Inside an > ORTE daemon? > > > On Feb 10, 2013, at 1:41 PM, Brice Goglin <brice.g

Re: [OMPI users] how to find the binding of each rank on the local machine

2013-02-10 Thread Brice Goglin
ity_str" for details (assuming you included the OMPI man > pages in your MANPATH), or look at it online at > > http://www.open-mpi.org/doc/v1.6/man3/OMPI_Affinity_str.3.php > > Remember, you have to configure with --enable-mpi-ext in order to enable the > extensions. >

Re: [OMPI users] how to find the binding of each rank on the local machine

2013-02-10 Thread Brice Goglin
I've been talking with Kranthi offline, he wants to use locality info inside OMPI. He needs the binding info from *inside* MPI. From 10 thousands feet, it looks like communicator->rank[X]->locality_info as a hwloc object or as a hwloc bitmap. Brice Le 10/02/2013 06:07, Ralph Castain a écrit :

Re: [hwloc-users] hwloc-bind --get on Solaris for binding to a single core

2013-02-08 Thread Brice Goglin
Le 07/02/2013 18:48, Eugene Loh a écrit : > I'm attaching a patch. Thanks a lot Eugene. I've tested that and it looks OK to me. I am committing it, it will be in v1.7 and v1.6.2. I guess now you want Jeff to include r5295 in OMPI. Brice

Re: [hwloc-users] How to get the coreid of a process on which it is currently mapped?

2013-02-06 Thread Brice Goglin
Hello I assume you're talking about MPI ranks, right? In the general case, you can't. hwloc knows about processes, not about ranks. And there's no standard way to convert between them. That said, Open MPI defines the OMPI_COMM_WORLD_RANK environment variable that contains what you want. If you

Re: [hwloc-users] OpenGL GPU detection code

2013-02-01 Thread Brice Goglin
I just committed big changes to the display branch (and I also merged latest trunk changes). lstopo will now report things like this: PCI 10de:06d1 GPU L#0 ":0.0" GPU L#1 "cuda0" GPU L#2 "nvml0" The changes include: 1) We don't have a "display" specific OS

Re: [hwloc-users] OpenGL GPU detection code

2013-01-30 Thread Brice Goglin
Le 29/01/2013 17:15, Stefan Eilemann a écrit : > On 29. Jan 2013, at 17:07, Brice Goglin <brice.gog...@inria.fr> wrote: > >> The current plan is to have some OpenCL device info and some OpenGL >> device info appear inside GPU PCI devices. That said, I am still not >&g

Re: [hwloc-users] OpenGL GPU detection code

2013-01-29 Thread Brice Goglin
gt; Sent: Tuesday, January 29, 2013 7:46 AM > To: Brice Goglin > Cc: Hardware locality user list > Subject: Re: [hwloc-users] OpenGL GPU detection code > > Hi Brice, > > On 29. Jan 2013, at 15:25, Brice Goglin <brice.gog...@inria.fr> wrote: > >> FWIW, I finally mana

Re: [hwloc-users] OpenGL GPU detection code

2013-01-29 Thread Brice Goglin
Le 29/01/2013 10:14, Stefan Eilemann a écrit : > Hi, > > On 29. Jan 2013, at 8:13, Brice Goglin <brice.gog...@inria.fr> wrote: > > [snip] >> Could you send a diff against this branch instead? > I missed this branch. I'll merge this into the current implementation and

Re: [hwloc-users] OpenGL GPU detection code

2013-01-29 Thread Brice Goglin
Le 28/01/2013 09:53, Stefan Eilemann a écrit : > Bonjour Brice, > > we've been dragging our feet a bit to get the GPU detection code merged into > upstream, and I'm trying to get back to this now. Since we fixed a minor > issue, I've merged our changes against the current svn trunk and opened a

Re: [hwloc-users] hwloc tutorial material

2013-01-22 Thread Brice Goglin
Le 22/01/2013 10:27, Samuel Thibault a écrit : > Kenneth A. Lloyd, le Mon 21 Jan 2013 22:46:37 +0100, a écrit : >> Thanks for making this tutorial available. Using hwloc 1.7, how far down >> into, say, NVIDIA cards can the architecture be reflected? Global memory >> size? SMX cores? None of the

Re: [hwloc-users] hwloc-1.6.1rc2 Build failure with Cray compiler

2013-01-17 Thread Brice Goglin
Does Cray fix such bugs quickly usually? If so, no need to change hwloc. If not, I'll need somebody to test the change on other cray platforms and compiler versions. Brice Jeff Hammond a écrit : >This is a bug in the Cray compiler. They cannot and should not set >the

Re: [hwloc-users] hwloc-1.6.1rc2 Build failure with Cray compiler

2013-01-17 Thread Brice Goglin
Did this work in the past? I don't think we changed this code recently. Can you run "make check" to make sure this change doesn't break anything? Thanks Brice Le 17/01/2013 19:19, Erik Schnetter a écrit : > hwloc-1.6.1rc2 fails to build with the Cray compiler > > Cray C : Version 8.1.2 Thu Jan

Re: [hwloc-users] Segmentation fault in collect_proc_cpuset, topology.c line 1074

2013-01-16 Thread Brice Goglin
> Olivier Cessenat. > > - Mail original - > De: "Brice Goglin" <brice.gog...@inria.fr> > À: "Hardware locality user list" <hwloc-us...@open-mpi.org>, cesse...@free.fr > Envoyé: Mardi 15 Janvier 2013 19:26:30 > Objet: Re: [hwloc-users] Segmentation faul

Re: [hwloc-users] Segmentation fault in collect_proc_cpuset, topology.c line 1074

2013-01-15 Thread Brice Goglin
Hello Indeed, there's a big cgroup crash in 1.6. Can you verify that 1.6.1rc2 works fine? Thanks Brice cesse...@free.fr a écrit : >Hello, > >When updating from 1.5.1 to 1.6 I get a segfault when inside a >cgroup/cpuset in collect_proc_cpuset, file topology.c line 1074. > >It appears that an

Re: [hwloc-users] Mapping a GPU to a pci local CPU on Windows

2013-01-08 Thread Brice Goglin
Is your machine NUMA? Maybe Windows returns an error when requesting numa info on non-NUMA info? Brice Le 08/01/2013 18:44, Ashley Reid a écrit : > OS says DEVPKEY_Numa_Proximity_Domain does not exist. Neither does > DEVPKEY_Device_Numa_Node . For all devices. > > Lame :/ > > Thanks, > Ash >

Re: [hwloc-users] hwloc on Blue Gene/Q?

2013-01-08 Thread Brice Goglin
Hello Erik, We need specific BGQ binding support, the binding API is different. Also we don't properly detect the 16 4-way cores properly, we only only 64 identical PUs. I am supposed to get a BGQ account in the near future so I hope I will have everything working in v1.7. Stay tuned Brice Le

Re: [hwloc-users] "-lnuma" missing from pkg-config information?

2013-01-04 Thread Brice Goglin
Hello Erik I am not a pkgconfig expert but my feeling is that this has buggy for a long time. hwloc.pc.in should likely use HWLOC_LIBS instead of LIBS. On my machine, it makes Libs.private change from -dl to -lm -lnuma here (with -lpci -lxml2 depending on the config). We also need to check whether

Re: [OMPI users] Error in configuring hwloc(hardware locality) on Linux on System Z

2012-12-13 Thread Brice Goglin
Le 13/12/2012 10:45, Shikha Maheshwari a écrit : > Hi, > > We are trying to build 'hwloc 1.4.2' on Linux on System Z. To build hwloc Hello, If you are really talking about hwloc, you should contact this mailing list Hardware locality user list (Open MPI and hwloc

Re: [OMPI users] running "openmpi" with "knem"

2012-12-01 Thread Brice Goglin
Le 01/12/2012 12:45, Leta Melkamu a écrit : > Hello there, > > I have some doubts on the use of knem with openmpi, everything works fine. > However, it is a bit not clear on the usage of knem flags while > running my open-mpi program. > Something like --mca btl_sm_knem_dma_min 4860 is enough or

Re: [hwloc-users] GPU devices appear as PCI devices

2012-11-20 Thread Brice Goglin
Ask nvidia to make their cuda driver free, and to add them to sysfs :) There's a cuda hwloc branch that will solve this. In the meantime, there are no nvidia osdevs. Maybe look at hwloc/cuda.h and cudart.h, they give cuda device affinity without osdevs. Brice Guillermo Miranda

Re: [hwloc-users] Windows api threading functions equivalent to hwloc?

2012-11-19 Thread Brice Goglin
Le 19/11/2012 21:01, Andrew Somorjai a écrit : > Below I posted a simple windows thread creation C++ routine which sets > the processor affinity to two cores. > What I want is the equivalent code using hwloc. Sorry for being > somewhat new to this but I'm not sure what > api calls are equivalent

Re: [hwloc-users] How do I access CPUModel info string

2012-11-18 Thread Brice Goglin
Le 26/10/2012 09:39, Brice Goglin a écrit : > Le 26/10/2012 05:22, Robin Scher a écrit : >> I would love to get this by my next release, say in the next 3-6 >> months. Is that something that would be possible? Is there anything I >> can do to help? > > We'll have a

Re: [hwloc-users] [hwloc-announce] Hardware locality (hwloc) v1.6rc1 released

2012-11-15 Thread Brice Goglin
Thanks, that was an old bug on a somehow rare XML case on a NUIOA machine. Looks like adding new test cases is indeed useful :) Brice Le 15/11/2012 13:14, Samuel Thibault a écrit : > Hello, > > Brice Goglin, le Tue 13 Nov 2012 13:45:28 +0100, a écrit : >> The Hardware Locali

Re: [OMPI users] mpi_leave_pinned is dangerous

2012-11-08 Thread Brice Goglin
My understanding of the upstreaming failure was more like: * Linus was going to be OK * Some perf (or trace?) guys came late and said "oh your code should be integrated into our more general stuff" but they didn't do it, and basically vetoed anything that didn't do what they said Brice Le

Re: [OMPI users] How is hwloc used by OpenMPI

2012-11-07 Thread Brice Goglin
Le 07/11/2012 21:26, Jeff Squyres a écrit : > On Nov 7, 2012, at 1:33 PM, Blosch, Edwin L wrote: > >> I see hwloc is a subproject hosted under OpenMPI but, in reading the >> documentation, I was unable to figure out if hwloc is a module within >> OpenMPI, or if some of the code base is borrowed

Re: [OMPI users] Best way to map MPI processes to sockets?

2012-11-07 Thread Brice Goglin
What processor and kernel is this? (see /proc/cpuinfo, or run "lstopo -v" and look for attributes on the Socket line) You're hwloc output looks like an Intel Xeon Westmere-EX (E7-48xx or E7-88xx). The likwid output is likely wrong (maybe confused by the fact that hardware threads are disabled).

[hwloc-users] hwloc@SC12

2012-11-07 Thread Brice Goglin
Hello, If you're attending SC12, feel free to come to the Inria booth (#1209) and say hello. Samuel and I will be there, happy to meet people in real life. Brice

Re: [hwloc-users] Strange binding issue on 40 core nodes and cgroups

2012-11-05 Thread Brice Goglin
Le 05/11/2012 22:57, Brock Palen a écrit : > Ok more information (had to build newer hwloc) My job today only 2 processes > are running at half speed and they indeed are sharing the same core: > > [root@nyx7000 ~]# for x in `cat /tmp/pids `; do echo -n "$x "; hwloc-bind >

Re: [hwloc-users] Strange binding issue on 40 core nodes and cgroups

2012-11-02 Thread Brice Goglin
Le 02/11/2012 21:22, Brice Goglin a écrit : > hwloc-bind --get-last-cpu-location --pid should give the same > info but it seems broken on my machine right now, going to debug. Actually, that works fine once you try it on a non-multithreaded program that uses all cores :) So you can u

Re: [hwloc-users] Strange binding issue on 40 core nodes and cgroups

2012-11-02 Thread Brice Goglin
Le 02/11/2012 21:03, Brock Palen a écrit : > This isn't a hwloc problem exactly, but maybe you can shed some insight. > > We have some 4 socket 10 core = 40 core nodes, HT off: > > depth 0: 1 Machine (type #1) > depth 1: 4 NUMANodes (type #2) > depth 2:4 Sockets (type #3) >

Re: [hwloc-users] How do I access CPUModel info string

2012-10-27 Thread Brice Goglin
int the number of sockets. of > http://www.open-mpi.org/projects/hwloc/doc/v1.5.1/ > ] > I see objects type 1,2,4 and 6 only. > > So, will there be another (non socket hwloc object based) way to get > CPUModel or will it find sockets as on Linux ? > > Thanks. > > Olivier Ce

Re: [hwloc-users] How do I access CPUModel info string

2012-10-26 Thread Brice Goglin
Le 26/10/2012 05:22, Robin Scher a écrit : > I would love to get this by my next release, say in the next 3-6 > months. Is that something that would be possible? Is there anything I > can do to help? We'll have a v1.6 release before the end of the year, and hopefully a first release candidate by

Re: [hwloc-users] How do I access CPUModel info string

2012-10-25 Thread Brice Goglin
Le 25/10/2012 23:57, Robin Scher a écrit : > On OS-X, you can get this string from the sysctlbyname() call: > > const char *name = "machdep.cpu.brand_string"; > char buffer[ 64 ]; > size_t size = 64; > if( !sysctlbyname( name, buffer, , NULL, 0 ) ) > memcpy( cpu_model,

Re: [hwloc-users] How do I access CPUModel info string

2012-10-25 Thread Brice Goglin
Le 25/10/2012 23:42, Samuel Thibault a écrit : > Robin Scher, le Thu 25 Oct 2012 23:39:46 +0200, a écrit : >> Is there a way to get this string (e.g. "Intel(R) Core(TM) i7 CPU M 620 @ >> 2.67GHz") consistently on Windows, Linux, OS-X and Solaris? > Currently, no. > > hwloc itself does not have a

Re: [hwloc-users] hwloc 1.5, freebsd and linux output on the same hardware

2012-10-03 Thread Brice Goglin
Le 03/10/2012 17:23, Sebastian Kuzminsky a écrit : > On Tue, Oct 2, 2012 at 5:14 PM, Samuel Thibault > > wrote: > > There were two bugs which resulted into cpuid not being properly > compiled. I have fixed them in the trunk, could

Re: [hwloc-users] hwloc 1.5, freebsd and linux output on the same hardware

2012-10-02 Thread Brice Goglin
Le 02/10/2012 23:45, Sebastian Kuzminsky a écrit : > Hi folks, I just discovered hwloc and it's really cool. Very useful, > so thanks! > > I'm trying to understand the hardware layout of a computer I'm working > with, an HP Proliant DL360p G8 server with two Intel E5-2690 processors. > > I'm

Re: [hwloc-users] Questions to lstopo and hwloc-bind

2012-09-14 Thread Brice Goglin
Le 14/09/2012 07:48, Siegmar Gross a écrit : > I have installed hwloc-1.5 on our systems and get the following output > when I run "lstopo" on a Sun Server M4000 (two quad-core processors with > two hardware-threads each). > > rs0 fd1026 101 lstopo > Machine (32GB) + NUMANode L#0 (P#1 32GB) >

Re: [hwloc-users] Solaris and hwloc

2012-09-13 Thread Brice Goglin
If the user really wants something to > run without binding, then you can just do that in the shell: > > - > hwloc-bind ...whatever... my_executable > if test "$?" != "0"; then > # run without binding > my_executable > fi > - > > My

Re: [hwloc-users] Solaris and hwloc

2012-09-13 Thread Brice Goglin
(resending because the formatting was bad) Le 13/09/2012 00:26, Jeff Squyres a écrit : > On Sep 12, 2012, at 10:30 AM, Samuel Thibault wrote: > >>> Sidenote: if hwloc-bind fails to bind, should we still launch the child >>> process? >> Well, it's up to you to decide :) > > Anyone have an

Re: [hwloc-users] Solaris and hwloc

2012-09-13 Thread Brice Goglin
Le 13/09/2012 00:26, Jeff Squyres a écrit : > On Sep 12, 2012, at 10:30 AM, Samuel Thibault wrote: > >>> Sidenote: if hwloc-bind fails to bind, should we still launch the child >>> process? >> Well, it's up to you to decide :) > > Anyone have an opinion? I'm 60/40 in favor of not letting it run,

Re: [OMPI users] [omx-devel] Open-mx issue with ompi 1.6.1

2012-09-12 Thread Brice Goglin
e problem is that the MTL component calls ompi_common_mx_initialize() only once in component_init() but it calls finalize() twice: once in component_close() and once in ompi_mtl_mx_finalize(). The attached patch seems to work. Signed-off-by: Brice Goglin <brice.gog...@inria.fr> Brice diff --git a/

<    1   2   3   4   5   >