Re: [hwloc-devel] PCI device name question

2012-03-22 Thread Christopher Samuel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 21/03/12 08:07, Brice Goglin wrote: > New patch attached, it doesn't add port numbers for non-IB > devices. Extract from lstopo on SGI XE270 box with Mellanox dual port IB card: PCIBridge PCI 15b3:673c Net L#2 "ib1" Net

Re: [hwloc-devel] PCI device name question

2012-03-20 Thread Jeffrey Squyres
On Mar 20, 2012, at 5:30 PM, Brice Goglin wrote: >> Does the new patch add port numbers at all if /device/infiniband >> doesn't exist? > > No. For each ethX, the hwloc "ethX" object will only get a Port number > if the corresponding sysfs device has some infiniband "child". > Otherwise, no Port

Re: [hwloc-devel] PCI device name question

2012-03-20 Thread Brice Goglin
Le 20/03/2012 22:12, Jeffrey Squyres a écrit : > On Mar 20, 2012, at 5:07 PM, Brice Goglin wrote: > >> New patch attached, it doesn't add port numbers for non-IB devices. > > Does the new patch add port numbers at all if /device/infiniband doesn't > exist? No. For each ethX, the hwloc "ethX" obje

Re: [hwloc-devel] PCI device name question

2012-03-20 Thread Jeffrey Squyres
On Mar 20, 2012, at 5:07 PM, Brice Goglin wrote: > New patch attached, it doesn't add port numbers for non-IB devices. Does the new patch add port numbers at all if /device/infiniband doesn't exist? I.e., is the dev_id/port number irrelevant if it's not an OpenFabrics device? -- Jeff Squyre

Re: [hwloc-devel] PCI device name question

2012-03-20 Thread Brice Goglin
Le 20/03/2012 21:48, Jeffrey Squyres a écrit : > On Mar 20, 2012, at 3:45 PM, Brice Goglin wrote: > >> That looks good to me, as long as starting port numbers to 1 for >> non-IB/OFED is OK. > > Hmm. Not sure about that. I always thought it was strange that IB devices > started with port 1. > >

Re: [hwloc-devel] PCI device name question

2012-03-20 Thread Brice Goglin
Le 20/03/2012 21:48, Jeffrey Squyres a écrit : > On Mar 20, 2012, at 3:45 PM, Brice Goglin wrote: > >> That looks good to me, as long as starting port numbers to 1 for >> non-IB/OFED is OK. > > Hmm. Not sure about that. I always thought it was strange that IB devices > started with port 1. > >

Re: [hwloc-devel] PCI device name question

2012-03-20 Thread Jeffrey Squyres
On Mar 20, 2012, at 3:45 PM, Brice Goglin wrote: > That looks good to me, as long as starting port numbers to 1 for > non-IB/OFED is OK. Hmm. Not sure about that. I always thought it was strange that IB devices started with port 1. Are *we* (hwloc) supplying the port number, or are you get

Re: [hwloc-devel] PCI device name question

2012-03-20 Thread Brice Goglin
Le 20/03/2012 20:30, Jeffrey Squyres a écrit : > On Mar 20, 2012, at 3:29 PM, Brice Goglin wrote: > >> By the way, do you want Port numbers to start at 0 or 1? > > IIRC, IB (and probably RoCE) port numbers start with 1. Shrug. > > So let's report whatever they report. > > The sample output you sho

Re: [hwloc-devel] PCI device name question

2012-03-20 Thread Jeffrey Squyres
On Mar 20, 2012, at 3:29 PM, Brice Goglin wrote: > By the way, do you want Port numbers to start at 0 or 1? IIRC, IB (and probably RoCE) port numbers start with 1. Shrug. So let's report whatever they report. The sample output you showed looks perfect to me. Is your patch small enough to por

Re: [hwloc-devel] PCI device name question

2012-03-20 Thread Brice Goglin
Le 20/03/2012 20:12, Jeffrey Squyres a écrit : > On Mar 20, 2012, at 3:09 PM, Brice Goglin wrote: > >> Looks like we just need to read /sys/class/net/ib*/dev_id, make that >> decimal, add one, and we get the port number. >> >> How would you like this to appear in the topology? Is a object info such

Re: [hwloc-devel] PCI device name question

2012-03-20 Thread Jeffrey Squyres
On Mar 20, 2012, at 3:09 PM, Brice Goglin wrote: > Looks like we just need to read /sys/class/net/ib*/dev_id, make that > decimal, add one, and we get the port number. > > How would you like this to appear in the topology? Is a object info such > as "Port=%d" in each network interface in a OFED o

Re: [hwloc-devel] PCI device name question

2012-03-20 Thread Brice Goglin
Le 20/03/2012 19:02, Jeffrey Squyres a écrit : > On Mar 20, 2012, at 12:02 PM, Brice Goglin wrote: > >> Actually, what we don't know is how to map that to port 1/2 (we have >> ib0/ib1 mac addresses, those are = GUID+1/2 on my machine) > > Yes, that is more correctly stated. > > A Mellanox guy point

Re: [hwloc-devel] PCI device name question

2012-03-20 Thread Jeffrey Squyres
On Mar 20, 2012, at 12:02 PM, Brice Goglin wrote: > Actually, what we don't know is how to map that to port 1/2 (we have > ib0/ib1 mac addresses, those are = GUID+1/2 on my machine) Yes, that is more correctly stated. A Mellanox guy pointed me to the ibdev2netdev script in current OFED distrib

Re: [hwloc-devel] PCI device name question

2012-03-20 Thread Brice Goglin
Le 20/03/2012 16:52, Jeffrey Squyres a écrit : > On Mar 20, 2012, at 11:09 AM, Brice Goglin wrote: > >>> I looked at that. It's not immediately clear to me how to map that to >>> "ib0" and "ib1", for example. >> Also I see that /sys/class/infiniband/mlx4_0/node_guid contains >> 0002:c903:000b:534

Re: [hwloc-devel] PCI device name question

2012-03-20 Thread Jeffrey Squyres
On Mar 20, 2012, at 11:09 AM, Brice Goglin wrote: >> I looked at that. It's not immediately clear to me how to map that to "ib0" >> and "ib1", for example. > > Also I see that /sys/class/infiniband/mlx4_0/node_guid contains > 0002:c903:000b:5348, that's ib0's end-of-address minus one, and ib1's

Re: [hwloc-devel] PCI device name question

2012-03-20 Thread Brice Goglin
Le 20/03/2012 15:53, Jeffrey Squyres a écrit : > On Mar 20, 2012, at 10:46 AM, Brice Goglin wrote: > >>> Is there a way in the hwloc topology data to tell which port eth0 and eth1 >>> correspond to? >> You should have a "Address" info attribute in each eth object containing >> something like 80:34

Re: [hwloc-devel] PCI device name question

2012-03-20 Thread Jeffrey Squyres
On Mar 20, 2012, at 10:46 AM, Brice Goglin wrote: >> Is there a way in the hwloc topology data to tell which port eth0 and eth1 >> correspond to? > > You should have a "Address" info attribute in each eth object containing > something like 80:34:00:48:fe:80:00:00:00:00:00:01:00:02:c9:03:00:0b:53

Re: [hwloc-devel] PCI device name question

2012-03-20 Thread Brice Goglin
Le 20/03/2012 15:33, Jeffrey Squyres a écrit : > On a machine I have, I'm getting output like this with hwloc trunk: > > PCIBridge > PCI 15b3:6750 > Net L#11 "eth0" > Net L#12 "eth1" > OpenFabrics L#13 "mlx4_0" > > which is all well and good (mlx4_0 is a RoCE card)

[hwloc-devel] PCI device name question

2012-03-20 Thread Jeffrey Squyres
On a machine I have, I'm getting output like this with hwloc trunk: PCIBridge PCI 15b3:6750 Net L#11 "eth0" Net L#12 "eth1" OpenFabrics L#13 "mlx4_0" which is all well and good (mlx4_0 is a RoCE card). Is there a way in the hwloc topology data to tell which port