On Fri, 2008-04-04 at 10:36 -0400, Brian J. Murrell wrote: > I'm trying to get a few nodes here connected with IPoIB. On the first > node I have tried with, after ifconfig'ing the interface into the > network with other IPoIB nodes I cannot seem to ping any other nodes. I > ran ibdiagnet and got a /tmp/ibdiagnet.pkey file with the following > contents: > > sata14:/ # cat /tmp/ibdiagnet.pkey > GROUP PKey:0x7fff Hosts:4 > Full sata15/P2 lid=0x0004 guid=0x00066a01a0000363 dev=23108 > Full sata14/P2 lid=0x0006 guid=0x00066a01a00002bf dev=23108 > Full sata23/P2 lid=0x0008 guid=0x00066a01a00002fe dev=23108 > Full sata16/P2 lid=0x0007 guid=0x00066a01a00002c1 dev=23108 > > When I run an "ibdiagpath -l 0x0004" I get the following: > > -W- Topology file is not specified. > Reports regarding cluster links will use direct routes. > -I- Using port 2 as the local port. > > -I--------------------------------------------------- > -I- Traversing the path from local to destination > -I--------------------------------------------------- > -I- From: lid=0x0006 guid=0x00066a01a00002bf dev=23108 sata14/P2 > -I- To: lid=0x0001 guid=0x00066a00c8000180 dev=5 Port=1 > > -I- From: lid=0x0001 guid=0x00066a00c8000180 dev=5 Port=2 > -I- To: lid=0x0004 guid=0x00066a01a0000363 dev=23108 sata15/P2 > > > -I--------------------------------------------------- > -I- PM Counters Info > -I--------------------------------------------------- > -I- No illegal PM counters values were found > > -I--------------------------------------------------- > -I- Path Partitions Report > -I--------------------------------------------------- > -I- Source sata14/P2 lid=0x0006 guid=0x00066a01a00002bf dev=23108 Port 2 > PKeys:0xffff > -I- Destination sata15 lid=0x0004 guid=0x00066a01a0000363 dev=23108 > PKeys:0xffff > -I- Path shared PKeys: 0xffff > > -I--------------------------------------------------- > -I- IPoIB Path Check > -I--------------------------------------------------- > -I- Subnet: IPv4 PKey:0x7fff QKey:0x00000000 MTU:2048Byte rate:10Gbps SL:0x00 > -W- Port sata14/P2 lid=0x0006 guid=0x00066a01a00002bf dev=23108 can not join > due > to rate:2.5Gbps < group:10Gbps > -W- Port sata15/P2 lid=0x0004 guid=0x00066a01a0000363 dev=23108 can not join > due > to rate:2.5Gbps < group:10Gbps > -E- No IPoIB Subnets found on Path! Nodes can not communicate via IPoIB! > > -I--------------------------------------------------- > -I- QoS on Path Check > -I--------------------------------------------------- > -W- Blocked VLs:4 5 at node:sata14 lid=0x0006 guid=0x00066a01a00002bf > dev=23108 > port:2 > -W- Blocked VLs:4 5 at node: lid=0x0001 guid=0x00066a00c8000180 dev=5 port:2 > -I- The following SLs can be used:0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 > > -I- Done. Run time was 0 seconds. > > That IPoIB Path Check looks a bit alarming. > > Anyone have any suggestions?
Looks like you have a mixed rate set of ports so you need to configure the group to 2.5 Gbps. What SM are you using ? -- Hal > b. > > > _______________________________________________ > general mailing list > [email protected] > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
