tests using connected mode IPoIB dmesg from the compute node cfd-cnsl-0001:~ # dmesg ib0: Start path record lookup for fe80:0000:0000:0000:00e0:8111:0100:0091 MTU > 0 ib0: PathRec LID 0x0518 for GID fe80:0000:0000:0000:00e0:8111:0100:0091 ib0: Created ah ffff810216a8c740 ib0: created address handle ffff81041c6533c0 for LID 0x0518, SL 0 ib0: Request connection 0x4a for gid fe80:0000:0000:0000:00e0:8111:0100:0091 qpn 0x404 ib0: REP received. ib0: REQ arrived ib0: Send unicast ARP to 0518 ib0: Send unicast ARP to 0518 ib0: Send unicast ARP to 0518 ib0: Send unicast ARP to 0518 ib0: Send unicast ARP to 0518 ib0: Start path record lookup for fe80:0000:0000:0000:00e0:8111:0100:007d MTU > 0 ib0: PathRec LID 0x0161 for GID fe80:0000:0000:0000:00e0:8111:0100:007d ib0: Created ah ffff810216a8c580 ib0: created address handle ffff81041e564d40 for LID 0x0161, SL 0 ib0: Send unicast ARP to 0161 ib0: REQ arrived ib0: Send unicast ARP to 0161 ib0: REQ arrived ib0: Send unicast ARP to 0161 ib0: Send unicast ARP to 0518 ib0: REQ arrived ib0: Send unicast ARP to 0518 ib0: Send unicast ARP to 0518 ib0: Send unicast ARP to 0161 ib0: REQ arrived ib0: DREQ received. ib0: CM error 9. ib0: Destroy active connection 0x4a head 0x19680 tail 0x19680
dmesg from the io node [EMAIL PROTECTED] ~]# dmesg ib0: Start path record lookup for fe80:0000:0000:0000:0002:c903:0000:c36d MTU > 0 ib0: PathRec LID 0x0384 for GID fe80:0000:0000:0000:0002:c903:0000:c36d ib0: Created ah ffff810126e93500 ib0: created address handle ffff81012b98cc80 for LID 0x0384, SL 0 ib0: Send unicast ARP to 0384 ib0: REQ arrived ib0: Request connection 0x10406 for gid fe80:0000:0000:0000:0002:c903:0000:c36d qpn 0x48 ib0: REP received. ib0: Send unicast ARP to 0384 ib0: Send unicast ARP to 0384 ib0: Send unicast ARP to 0384 ib0: Send unicast ARP to 0384 ib0: Send unicast ARP to 0384 ib0: Send unicast ARP to 045e ib0: REQ arrived ib0: neigh_destructor for bonding device: ib0 ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:c36d ib0: Reap connection for gid fe80:0000:0000:0000:0002:c903:0000:c36d ib0: Destroy active connection 0x10406 head 0x12455 tail 0x12455 ib0: Request connection 0x30406 for gid fe80:0000:0000:0000:0002:c903:0000:cad1 qpn 0x48 ib0: REP received. ib0: Request connection 0x30407 for gid fe80:0000:0000:0000:0002:c903:0000:c36d qpn 0x48 ib0: REP received. ib0: neigh_destructor for bonding device: ib0 ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:cad1 ib0: Reap connection for gid fe80:0000:0000:0000:0002:c903:0000:cad1 ib0: Destroy active connection 0x30406 head 0x8 tail 0x8 ib0: Request connection 0x40406 for gid fe80:0000:0000:0000:0002:c903:0000:cad1 qpn 0x48 ib0: REP received. ib0: Send unicast ARP to 0384 ib0: Send unicast ARP to 0384 ib0: neigh_destructor for bonding device: ib0 ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:cad1 ib0: Reap connection for gid fe80:0000:0000:0000:0002:c903:0000:cad1 ib0: Destroy active connection 0x40406 head 0x8 tail 0x8 ib0: neigh_destructor for bonding device: ib0 ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:c36d ib0: Reap connection for gid fe80:0000:0000:0000:0002:c903:0000:c36d ib0: Destroy active connection 0x30407 head 0x3 tail 0x3 ib0: Send unicast ARP to 045e ib0: REQ arrived ib0: Request connection 0x80405 for gid fe80:0000:0000:0000:0002:c903:0000:cad1 qpn 0x48 ib0: REQ arrived ib0: REP received. ib0: Send unicast ARP to 045e ib0: REQ arrived ib0: Send unicast ARP to 045e On Thu, Jul 10, 2008 at 9:21 AM, Michael Di Domenico <[EMAIL PROTECTED]> wrote: > tests using datagram IPoIB (non-connected mode) > dmesg from the compute node > cfd-cnsl-0001:~ # dmesg > ib0: Send unicast ARP to 0518 > ib0: Send unicast ARP to 0518 > ib0: Send unicast ARP to 0518 > ib0: Send unicast ARP to 0518 > ib0: Start path record lookup for fe80:0000:0000:0000:00e0:8111:0100:007d > MTU > 1024 > ib0: PathRec LID 0x0161 for GID fe80:0000:0000:0000:00e0:8111:0100:007d > ib0: Created ah ffff81042063dc80 > ib0: created address handle ffff8102206144c0 for LID 0x0161, SL 0 > ib0: Send unicast ARP to 0161 > ib0: Send unicast ARP to 0161 > ib0: Send unicast ARP to 0161 > ib0: Send unicast ARP to 0518 > ib0: Send unicast ARP to 0518 > ib0: Send unicast ARP to 0518 > ib0: Send unicast ARP to 0161 > ib0: neigh_destructor for bonding device: ib0 > ib0: neigh_cleanup for 000404 fe80:0000:0000:0000:00e0:8111:0100:0091 > > > dmesg from the IO node > > [EMAIL PROTECTED] ~]# dmesg > ib0: Send unicast ARP to 0384 > ib0: Send unicast ARP to 0384 > ib_mthca 0000:07:00.0: too many gathers > ib0: post_send failed > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib0: Send unicast ARP to 0384 > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib0: Send unicast ARP to 045e > ib0: REQ arrived > ib0: Send unicast ARP to 0384 > ib0: Send unicast ARP to 045e > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib0: Send unicast ARP to 0384 > ib0: neigh_destructor for bonding device: ib0 > ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:cad1 > ib0: neigh_destructor for bonding device: ib0 > ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:c36d > ib0: neigh_destructor for bonding device: ib0 > ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:cad1 > ib0: REQ arrived > ib0: neigh_destructor for bonding device: ib0 > ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:cad1 > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib0: neigh_destructor for bonding device: ib0 > ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:c36d > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib0: Send unicast ARP to 0384 > ib0: neigh_destructor for bonding device: ib0 > ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:c36d > ib0: Send unicast ARP to 045e > ib0: neigh_destructor for bonding device: ib0 > ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:cad1 > > > On Thu, Jul 10, 2008 at 8:33 AM, Eli Cohen <[EMAIL PROTECTED]> wrote: > >> On Thu, Jul 10, 2008 at 07:57:30AM -0400, Michael Di Domenico wrote: >> > maybe i spoke too soon, so more output came, i thought it was done >> > ib0: mtu > 2044 will cause multicast packet drops. >> > eth5: no IPv6 routers present >> > ib0: Send unicast ARP to 0384 >> > ib0: REQ arrived >> > ib0: Request connection 0x2c0406 for gid >> > fe80:0000:0000:0000:0002:c903:0000:c36d qpn 0x48 >> > ib0: REP received. >> > ib0: Send unicast ARP to 0384 >> > ib0: Send unicast ARP to 045e >> > ib0: REQ arrived >> > ib0: Send unicast ARP to 0384 >> > ib0: Send unicast ARP to 0384 >> > ib0: Send unicast ARP to 0384 >> > ib0: Send unicast ARP to 045e >> > ib0: REQ arrived >> > ib0: neigh_destructor for bonding device: ib0 >> > ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:c36d >> > ib0: Reap connection for gid fe80:0000:0000:0000:0002:c903:0000:c36d >> > ib0: Destroy active connection 0x2c0406 head 0x22644 tail 0x22644 >> > ib0: Request connection 0x2f0406 for gid >> > fe80:0000:0000:0000:0002:c903:0000:cad1 qpn 0x48 >> > ib0: REP received. >> > ib0: neigh_destructor for bonding device: ib0 >> > ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:cad1 >> > ib0: Reap connection for gid fe80:0000:0000:0000:0002:c903:0000:cad1 >> > ib0: Destroy active connection 0x2f0406 head 0x4 tail 0x4 >> >> >> I see you're working in connected mode. Can you please do the >> follwoing: >> >> 1. clear dmesg: dmesg -c >> 2. run again, then send all the output of dmesg >> >> Do this for both connected and datagram modes. >> >> Thanks. >> >> >
_______________________________________________ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general