I'm having intermittent problems with opensm.. It seems after a while
IPoIB stops working and if I restart opensm, it starts spitting out
errors. Do I have a misbehaving switch somewhere?

ibnetdiscover seems to work fine.


(this is from running 'opensm -v -o -r')


-------------------------------------------------
OpenSM Rev:openib-1.0.0
Command Line Arguments:
 Verbose option -v (log flags = 0x7)
 Run Once
 Reassign LIDs
-------------------------------------------------
Jun 02 18:14:35 [B7E7F600] -> OpenSM Rev:openib-1.0.0
Jun 02 18:14:35 [B7E7F600] -> osm_opensm_init: Forcing single threaded 
dispatcher.
Jun 02 18:14:35 [B7E7F600] -> osm_report_notice: Reporting Generic Notice 
type:3 num:66 from LID:0x0000 GID:0xfe80000000000000,0x0000000000000000
Jun 02 18:14:35 [B7E7F600] -> osm_report_notice: Reporting Generic Notice 
type:3 num:66 from LID:0x0000 GID:0xfe80000000000000,0x0000000000000000
Jun 02 18:14:35 [B7E7F600] -> osm_vendor_get_all_port_attr: assign CA mthca0 
port 1 guid (0x66a00a000044d) as the default port.
using default guid 0x66a00a000044d
Jun 02 18:14:35 [B7E7F600] -> osm_sm_mad_ctrl_bind: Binding to port 
0x66a00a000044d.
Jun 02 18:14:35 [B7E7F600] -> osm_vendor_bind: Binding to port 0x66a00a000044d.
Jun 02 18:14:35 [B7E7F600] -> osm_sa_mad_ctrl_bind: Binding to port GUID = 
0x66a00a000044d.
Jun 02 18:14:35 [B7E7F600] -> osm_vendor_bind: Binding to port 0x66a00a000044d.
Jun 02 18:14:35 [B7E7F600] -> __osm_sm_state_mgr_discovering_msg: 


******************************************************************
******************** ENTERING SM DISCOVERING STATE ***************
******************************************************************


Jun 02 18:14:35 [B7E7F600] -> __osm_state_mgr_sweep_heavy_msg: 


******************************************************************
******************** INITIATING HEAVY SWEEP **********************
******************************************************************


Jun 02 18:14:35 [B567DBB0] -> Received a SMP on a 0 hop path:
                                Initial path = [0]
                                Return path  = [0]
Jun 02 18:14:35 [B567DBB0] -> __osm_ni_rcv_process_new: Discovered new Channel 
Adapter node,
                                GUID = 0x66a009800044d, TID = 0x1234.
Jun 02 18:14:35 [B567DBB0] -> __osm_nd_rcv_process_nd: Node 0x66a009800044d
                                Description = MT23108 InfiniHost Mellanox 
Technologies.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 1 with 
GUID = 0x66a00a000044d for parent node GUID = 0x66a009800044d, TID = 0x1236.
Jun 02 18:14:35 [B567DBB0] -> osm_vendor_set_sm: setting IS_SM capability mask 
failed; errno 2
Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 
port_num 1 with GUID = 0x66a00a000044d for parent node GUID = 0x66a009800044d, 
TID = 0x1237.
Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:1 
port_num 1 with GUID = 0x66a00a000044d for parent node GUID = 0x66a009800044d, 
TID = 0x1238.
Jun 02 18:14:35 [B567DBB0] -> Received a SMP on a 1 hop path:
                                Initial path = [0][1]
                                Return path  = [0][4]
Jun 02 18:14:35 [B567DBB0] -> __osm_ni_rcv_process_new: Discovered new Switch 
node,
                                GUID = 0x84e000000014a, TID = 0x1239.
Jun 02 18:14:35 [B567DBB0] -> __osm_nd_rcv_process_nd: Node 0x84e000000014a
                                Description = DivergeNet Meshlink 4X Switch.
Jun 02 18:14:35 [B567DBB0] -> __osm_si_rcv_process_new:Subnet max multicast lid 
is 0xC400
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 0 with 
GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, TID = 0x123c.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x0.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 1 with 
GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, TID = 0x123d.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x1.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 2 with 
GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, TID = 0x123e.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x2.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 3 with 
GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, TID = 0x123f.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x3.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 4 with 
GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, TID = 0x1240.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 5 with 
GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, TID = 0x1241.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x5.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 6 with 
GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, TID = 0x1242.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x6.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 7 with 
GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, TID = 0x1243.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x7.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 8 with 
GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, TID = 0x1244.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x8.
Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 
port_num 0 with GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, 
TID = 0x1246.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x1
                                trans_id................0x1247
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x10000
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1]
                                Return path:  [0][4]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 
port_num 1 with GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, 
TID = 0x1247.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x1
                                trans_id................0x1248
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x10001
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1]
                                Return path:  [0][4]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:1 
port_num 1 with GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, 
TID = 0x1248.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x1
                                trans_id................0x1249
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x20000
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1]
                                Return path:  [0][4]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 
port_num 2 with GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, 
TID = 0x1249.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x1
                                trans_id................0x124a
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x20001
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1]
                                Return path:  [0][4]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:1 
port_num 2 with GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, 
TID = 0x124a.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x1
                                trans_id................0x124b
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x30000
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1]
                                Return path:  [0][4]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 
port_num 3 with GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, 
TID = 0x124b.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x1
                                trans_id................0x124c
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x30001
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1]
                                Return path:  [0][4]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:1 
port_num 3 with GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, 
TID = 0x124c.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x1
                                trans_id................0x124d
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x40000
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1]
                                Return path:  [0][4]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 
port_num 4 with GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, 
TID = 0x124d.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x1
                                trans_id................0x124e
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x40001
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1]
                                Return path:  [0][4]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:1 
port_num 4 with GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, 
TID = 0x124e.
Jun 02 18:14:35 [B567DBB0] -> Received a SMP on a 2 hop path:
                                Initial path = [0][1][5]
                                Return path  = [0][4][6]
Jun 02 18:14:35 [B567DBB0] -> __osm_ni_rcv_process_new: Discovered new Switch 
node,
                                GUID = 0x84e0000000152, TID = 0x124f.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x1
                                trans_id................0x1250
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x50000
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1]
                                Return path:  [0][4]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x1
                                trans_id................0x1251
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x50001
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1]
                                Return path:  [0][4]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x1
                                trans_id................0x1252
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x60000
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1]
                                Return path:  [0][4]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 
port_num 5 with GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, 
TID = 0x1250.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x1
                                trans_id................0x1253
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x60001
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1]
                                Return path:  [0][4]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:1 
port_num 5 with GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, 
TID = 0x1251.
Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 
port_num 6 with GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, 
TID = 0x1252.
Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:1 
port_num 6 with GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, 
TID = 0x1253.
Jun 02 18:14:35 [B567DBB0] -> Received a SMP on a 2 hop path:
                                Initial path = [0][1][7]
                                Return path  = [0][4][7]
Jun 02 18:14:35 [B567DBB0] -> __osm_ni_rcv_process_new: Discovered new Switch 
node,
                                GUID = 0x2c900001cee10, TID = 0x1254.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x1
                                trans_id................0x1255
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x70000
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1]
                                Return path:  [0][4]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 
port_num 7 with GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, 
TID = 0x1255.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x1
                                trans_id................0x1256
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x70001
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1]
                                Return path:  [0][4]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:1 
port_num 7 with GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, 
TID = 0x1256.
Jun 02 18:14:35 [B567DBB0] -> __osm_ni_rcv_process_existing: Rediscovered 
Switch node 0x2c900001cee10
                                TID = 0x1257, discovered 1 times already.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x1
                                trans_id................0x1258
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x80000
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1]
                                Return path:  [0][4]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 
port_num 8 with GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, 
TID = 0x1258.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x1
                                trans_id................0x1259
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x80001
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1]
                                Return path:  [0][4]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:1 
port_num 8 with GUID = 0x84e000000014a for parent node GUID = 0x84e000000014a, 
TID = 0x1259.
Jun 02 18:14:35 [B567DBB0] -> __osm_nd_rcv_process_nd: Node 0x84e0000000152
                                Description = DivergeNet Meshlink 4X Switch.
Jun 02 18:14:35 [B567DBB0] -> osm_si_rcv_process: ERR 3610: 
                                Bad LinearFDBTop value = 0xC000 on switch 
0x84e0000000152.
                                Forcing correction to 0x0.
Jun 02 18:14:35 [B567DBB0] -> __osm_nd_rcv_process_nd: Node 0x2c900001cee10
                                Description = MT43132 Mellanox Technologies.
Jun 02 18:14:35 [B567DBB0] -> __osm_si_rcv_process_existing: discovery_count 
is:2 
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 0 with 
GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, TID = 0x125f.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x0.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 1 with 
GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, TID = 0x1260.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x1.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 2 with 
GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, TID = 0x1261.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x2.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 3 with 
GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, TID = 0x1262.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x3.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 4 with 
GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, TID = 0x1263.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x4.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 5 with 
GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, TID = 0x1264.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x5.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 6 with 
GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, TID = 0x1265.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 7 with 
GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, TID = 0x1266.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x7.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 8 with 
GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, TID = 0x1267.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x8.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 0 with 
GUID = 0x2c900001cee10 for parent node GUID = 0x2c900001cee10, TID = 0x1269.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x0.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 1 with 
GUID = 0x2c900001cee10 for parent node GUID = 0x2c900001cee10, TID = 0x126a.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x1.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 2 with 
GUID = 0x2c900001cee10 for parent node GUID = 0x2c900001cee10, TID = 0x126b.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x2.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 3 with 
GUID = 0x2c900001cee10 for parent node GUID = 0x2c900001cee10, TID = 0x126c.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x3.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 4 with 
GUID = 0x2c900001cee10 for parent node GUID = 0x2c900001cee10, TID = 0x126d.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x4.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 5 with 
GUID = 0x2c900001cee10 for parent node GUID = 0x2c900001cee10, TID = 0x126e.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x5.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 6 with 
GUID = 0x2c900001cee10 for parent node GUID = 0x2c900001cee10, TID = 0x126f.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x6.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 7 with 
GUID = 0x2c900001cee10 for parent node GUID = 0x2c900001cee10, TID = 0x1270.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Discovered port num 8 with 
GUID = 0x2c900001cee10 for parent node GUID = 0x2c900001cee10, TID = 0x1271.
Jun 02 18:14:35 [B567DBB0] -> osm_pi_rcv_process: Initializing port number 0x8.
Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 
port_num 0 with GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, 
TID = 0x1273.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x2
                                trans_id................0x1274
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x10000
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1][5]
                                Return path:  [0][4][6]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x2
                                trans_id................0x1275
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x10001
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1][5]
                                Return path:  [0][4][6]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 
port_num 1 with GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, 
TID = 0x1274.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x2
                                trans_id................0x1276
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x20000
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1][5]
                                Return path:  [0][4][6]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:1 
port_num 1 with GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, 
TID = 0x1275.
Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 
port_num 2 with GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, 
TID = 0x1276.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x2
                                trans_id................0x1277
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x20001
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1][5]
                                Return path:  [0][4][6]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:1 
port_num 2 with GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, 
TID = 0x1277.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x2
                                trans_id................0x1278
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x30000
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1][5]
                                Return path:  [0][4][6]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 
port_num 3 with GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, 
TID = 0x1278.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x2
                                trans_id................0x1279
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x30001
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1][5]
                                Return path:  [0][4][6]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:1 
port_num 3 with GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, 
TID = 0x1279.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x2
                                trans_id................0x127a
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x40000
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1][5]
                                Return path:  [0][4][6]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 
port_num 4 with GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, 
TID = 0x127a.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x2
                                trans_id................0x127b
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x40001
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1][5]
                                Return path:  [0][4][6]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:1 
port_num 4 with GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, 
TID = 0x127b.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x2
                                trans_id................0x127c
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x50000
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1][5]
                                Return path:  [0][4][6]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 
port_num 5 with GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, 
TID = 0x127c.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x2
                                trans_id................0x127d
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x50001
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1][5]
                                Return path:  [0][4][6]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:1 
port_num 5 with GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, 
TID = 0x127d.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x2
                                trans_id................0x127e
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x60000
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1][5]
                                Return path:  [0][4][6]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 
port_num 6 with GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, 
TID = 0x127e.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x2
                                trans_id................0x127f
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x60001
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1][5]
                                Return path:  [0][4][6]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:1 
port_num 6 with GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, 
TID = 0x127f.
Jun 02 18:14:35 [B567DBB0] -> Received a SMP on a 3 hop path:
                                Initial path = [0][1][5][7]
                                Return path  = [0][4][6][1]
Jun 02 18:14:35 [B567DBB0] -> __osm_ni_rcv_process_new: Discovered new Channel 
Adapter node,
                                GUID = 0x2c90108cd8ba0, TID = 0x1280.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x2
                                trans_id................0x1281
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x70000
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1][5]
                                Return path:  [0][4][6]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 
port_num 7 with GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, 
TID = 0x1281.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x2
                                trans_id................0x1282
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x70001
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1][5]
                                Return path:  [0][4][6]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:1 
port_num 7 with GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, 
TID = 0x1282.
Jun 02 18:14:35 [B567DBB0] -> Received a SMP on a 3 hop path:
                                Initial path = [0][1][5][8]
                                Return path  = [0][4][6][1]
Jun 02 18:14:35 [B567DBB0] -> __osm_ni_rcv_process_new: Discovered new Channel 
Adapter node,
                                GUID = 0x2c90108cd8a50, TID = 0x1283.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x2
                                trans_id................0x1284
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x80000
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1][5]
                                Return path:  [0][4][6]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 
port_num 8 with GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, 
TID = 0x1284.
Jun 02 18:14:35 [B3E7DBB0] -> __osm_sm_mad_ctrl_rcv_callback: ERR 3111: Error 
status = 0x1C00.
Jun 02 18:14:35 [B3E7DBB0] -> SMP dump:
                                base_ver................0x1
                                mgmt_class..............0x81
                                class_ver...............0x1
                                method..................0x81 (SubnGetResp)
                                status..................0x801C
                                hop_ptr.................0x0
                                hop_count...............0x2
                                trans_id................0x1285
                                attr_id.................0x16 (P_KeyTable)
                                resv....................0x0
                                attr_mod................0x80001
                                m_key...................0x0000000000000000
                                dr_slid.................0xFFFF
                                dr_dlid.................0xFFFF

                                Initial path: [0][1][5]
                                Return path:  [0][4][6]
                                Reserved:     [0][0][0][0][0][0][0]

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

                                00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 
00

Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:1 
port_num 8 with GUID = 0x84e0000000152 for parent node GUID = 0x84e0000000152, 
TID = 0x1285.
Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 
port_num 0 with GUID = 0x2c900001cee10 for parent node GUID = 0x2c900001cee10, 
TID = 0x1286.
Jun 02 18:14:35 [B567DBB0] -> Received a SMP on a 3 hop path:
                                Initial path = [0][1][7][1]
                                Return path  = [0][4][7][1]
Jun 02 18:14:35 [B567DBB0] -> __osm_ni_rcv_process_new: Discovered new Channel 
Adapter node,
                                GUID = 0x66a009800043c, TID = 0x1287.
Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 
port_num 1 with GUID = 0x2c900001cee10 for parent node GUID = 0x2c900001cee10, 
TID = 0x1288.
Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:1 
port_num 1 with GUID = 0x2c900001cee10 for parent node GUID = 0x2c900001cee10, 
TID = 0x1289.
Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 
port_num 2 with GUID = 0x2c900001cee10 for parent node GUID = 0x2c900001cee10, 
TID = 0x128a.
Jun 02 18:14:35 [B567DBB0] -> osm_pkey_rcv_process: Got GetResp(PKey) block:1 
port_num 2 with GUID = 0x2c900001cee10 for parent node GUID = 0x2c900001cee10, 
TID = 0x128b.
Jun 02 18:14:35 [B3E7DBB0] -> umad_receiver: send completed with error 
(method=1 attr=11) -- dropping.
Jun 02 18:14:35 [B3E7DBB0] -> umad_receiver: DR SMP hop ptr 0 hop count 3 DR 
SLID 0xffff DR DLID 0xffff
Jun 02 18:14:35 [B3E7DBB0] -> Initial path = [1][81][1]
Jun 02 18:14:36 [B3E7DBB0] -> umad_receiver: send completed with error 
(method=1 attr=11) -- dropping.
Jun 02 18:14:36 [B3E7DBB0] -> umad_receiver: DR SMP hop ptr 0 hop count 3 DR 
SLID 0xffff DR DLID 0xffff
Jun 02 18:14:36 [B3E7DBB0] -> Initial path = [1][81][1]
Jun 02 18:14:36 [B3E7DBB0] -> umad_receiver: send completed with error 
(method=1 attr=11) -- dropping.
Jun 02 18:14:36 [B3E7DBB0] -> umad_receiver: DR SMP hop ptr 0 hop count 3 DR 
SLID 0xffff DR DLID 0xffff
Jun 02 18:14:36 [B3E7DBB0] -> Initial path = [1][81][1]
Jun 02 18:14:36 [B3E7DBB0] -> umad_receiver: send completed with error 
(method=1 attr=11) -- dropping.
Jun 02 18:14:36 [B3E7DBB0] -> umad_receiver: DR SMP hop ptr 0 hop count 3 DR 
SLID 0xffff DR DLID 0xffff
Jun 02 18:14:36 [B3E7DBB0] -> Initial path = [1][81][1]
Jun 02 18:14:36 [B3E7DBB0] -> umad_receiver: send completed with error 
(method=1 attr=11) -- dropping.
Jun 02 18:14:36 [B3E7DBB0] -> umad_receiver: DR SMP hop ptr 0 hop count 3 DR 
SLID 0xffff DR DLID 0xffff
Jun 02 18:14:36 [B3E7DBB0] -> Initial path = [1][81][1]
Jun 02 18:14:36 [B3E7DBB0] -> umad_receiver: send completed with error 
(method=1 attr=11) -- dropping.
Jun 02 18:14:36 [B3E7DBB0] -> umad_receiver: DR SMP hop ptr 0 hop count 3 DR 
SLID 0xffff DR DLID 0xffff
Jun 02 18:14:36 [B3E7DBB0] -> Initial path = [1][81][1]
Jun 02 18:14:36 [B3E7DBB0] -> umad_receiver: send completed with error 
(method=1 attr=11) -- dropping.
Jun 02 18:14:36 [B3E7DBB0] -> umad_receiver: DR SMP hop ptr 0 hop count 3 DR 
SLID 0xffff DR DLID 0xffff
Jun 02 18:14:36 [B3E7DBB0] -> Initial path = [1][81][1]
Jun 02 18:14:36 [B3E7DBB0] -> umad_receiver: send completed with error 
(method=1 attr=11) -- dropping.
Jun 02 18:14:36 [B3E7DBB0] -> umad_receiver: DR SMP hop ptr 0 hop count 3 DR 
SLID 0xffff DR DLID 0xffff
Jun 02 18:14:36 [B3E7DBB0] -> Initial path = [1][81][1]
Jun 02 18:14:36 [B3E7DBB0] -> umad_receiver: send completed with error 
(method=1 attr=11) -- dropping.
Jun 02 18:14:36 [B3E7DBB0] -> umad_receiver: DR SMP hop ptr 0 hop count 3 DR 
SLID 0xffff DR DLID 0xffff
Jun 02 18:14:36 [B3E7DBB0] -> Initial path = [1][81][1]
Jun 02 18:14:36 [B3E7DBB0] -> umad_receiver: send completed with error 
(method=1 attr=11) -- dropping.
Jun 02 18:14:36 [B3E7DBB0] -> umad_receiver: DR SMP hop ptr 0 hop count 3 DR 
SLID 0xffff DR DLID 0xffff
Jun 02 18:14:36 [B3E7DBB0] -> Initial path = [1][81][1]
Jun 02 18:14:36 [B3E7DBB0] -> umad_receiver: send completed with error 
(method=1 attr=11) -- dropping.
Jun 02 18:14:36 [B3E7DBB0] -> umad_receiver: DR SMP hop ptr 0 hop count 3 DR 
SLID 0xffff DR DLID 0xffff
Jun 02 18:14:36 [B3E7DBB0] -> Initial path = [1][81][1]
Jun 02 18:14:37 [B3E7DBB0] -> umad_receiver: send completed with error 
(method=1 attr=11) -- dropping.
Jun 02 18:14:37 [B3E7DBB0] -> umad_receiver: DR SMP hop ptr 0 hop count 3 DR 
SLID 0xffff DR DLID 0xffff
Jun 02 18:14:37 [B3E7DBB0] -> Initial path = [1][81][1]
Jun 02 18:14:37 [B3E7DBB0] -> umad_receiver: send completed with error 
(method=1 attr=11) -- dropping.
Jun 02 18:14:37 [B3E7DBB0] -> umad_receiver: DR SMP hop ptr 0 hop count 3 DR 
SLID 0xffff DR DLID 0xffff
Jun 02 18:14:37 [B3E7DBB0] -> Initial path = [1][81][1]
Jun 02 18:14:37 [B3E7DBB0] -> umad_receiver: send completed with error 
(method=1 attr=11) -- dropping.
Jun 02 18:14:37 [B3E7DBB0] -> umad_receiver: DR SMP hop ptr 0 hop count 3 DR 
SLID 0xffff DR DLID 0xffff
Jun 02 18:14:37 [B3E7DBB0] -> Initial path = [1][81][1]
_______________________________________________
openib-general mailing list
[email protected]
http://openib.org/mailman/listinfo/openib-general

To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general

Reply via email to