On Sat, 27 Jul 2013 at 08:35 -0000, Jarrod B Johnson wrote:
> How odd. This happens all the time or occasionally?
It happens a lot of the time. Different BMCs and for short periods
and then that BMC is back to normal.
> Can you do nodels bc081 ipmi.bmc
>
> Then, do nodels ipmi.bmc=<value from above>
# nodels bc081 ipmi.bmc
bc081: bc081-imm
# nodels ipmi.bmc=bc081-imm
bc081
#
> That means the BMC rejected our attempt to connect as it had already
> reached the maximum user count.
When it is happening I can't use ipmitool either. When it clears
'ipmitool session info all' shows 2 of 9 session slots in use (one is
conserver/sol and one is the ipmitool invocation).
Sometimes I catch a couple additional sessions:
# ipmitool -I lanplus -H bc059-imm -U USERID -P ****** session info all
session handle : 48
slot count : 9
active sessions : 4
user id : 2
privilege level : ADMINISTRATOR
session type : IPMIv2/RMCP+
channel number : 0x01
console ip : 172.18.240.41
console mac : e4:1f:13:2d:db:70
console port : 55811
session handle : 49
slot count : 9
active sessions : 4
user id : 2
privilege level : USER
session type : IPMIv2/RMCP+
channel number : 0x01
console ip : 172.18.240.41
console mac : e4:1f:13:2d:db:70
console port : 59309
session handle : 50
slot count : 9
active sessions : 4
user id : 2
privilege level : USER
session type : IPMIv2/RMCP+
channel number : 0x01
console ip : 172.18.240.41
console mac : e4:1f:13:2d:db:70
console port : 44407
session handle : 51
slot count : 9
active sessions : 4
user id : 2
privilege level : ADMINISTRATOR
session type : IPMIv2/RMCP+
channel number : 0x01
console ip : 172.18.240.50
console mac : 52:54:00:12:3f:cc
console port : 54006
session handle : 0
slot count : 9
active sessions : 4
session handle : 0
slot count : 9
active sessions : 4
session handle : 0
slot count : 9
active sessions : 4
session handle : 0
slot count : 9
active sessions : 4
session handle : 0
slot count : 9
active sessions : 4
#
Looking at tcpdump traces I can see some strange state changes...
Port 44407:
688 22.176307 172.18.240.41 172.18.243.59 IPMI/ATCA 65
Req, Get Channel Authentication Capabilities, seq 0x00
689 22.198550 172.18.243.59 172.18.240.41 IPMI/ATCA 72
Rsp, Get Channel Authentication Capabilities, seq 0x00
690 22.199331 172.18.240.41 172.18.243.59 RMCP+ 90
Session ID 0x0, payload type: RMCP+ Open Session Request
691 22.200848 172.18.243.59 172.18.240.41 RMCP+ 94
Session ID 0x0, payload type: RMCP+ Open Session Response
692 22.213244 172.18.240.41 172.18.243.59 RMCP+ 92
Session ID 0x0, payload type: RAKP Message 1
693 22.263772 172.18.243.59 172.18.240.41 RMCP+ 118
Session ID 0x0, payload type: RAKP Message 2
[timeout???]
694 23.063794 172.18.240.41 172.18.243.59 IPMI/ATCA 65
Req, Get Channel Authentication Capabilities, seq 0x00
695 23.088369 172.18.243.59 172.18.240.41 IPMI/ATCA 72
Rsp, Get Channel Authentication Capabilities, seq 0x00
696 23.104560 172.18.240.41 172.18.243.59 RMCP+ 90
Session ID 0x0, payload type: RMCP+ Open Session Request
697 23.105848 172.18.243.59 172.18.240.41 RMCP+ 94
Session ID 0x0, payload type: RMCP+ Open Session Response
698 23.106087 172.18.240.41 172.18.243.59 RMCP+ 92
Session ID 0x0, payload type: RAKP Message 1
699 23.155110 172.18.243.59 172.18.240.41 RMCP+ 118
Session ID 0x0, payload type: RAKP Message 2
700 23.155403 172.18.240.41 172.18.243.59 RMCP+ 86
Session ID 0x0, payload type: RAKP Message 3
701 23.171482 172.18.243.59 172.18.240.41 RMCP+ 78
Session ID 0x0, payload type: RAKP Message 4
702 23.171771 172.18.240.41 172.18.243.59 IPMI/ATCA 82
Req, Set Session Privilege Level, seq 0x02
703 23.187612 172.18.243.59 172.18.240.41 IPMI/ATCA 82
Rsp, Set Session Privilege Level, seq 0x02
704 23.188419 172.18.240.41 172.18.243.59 IPMI/ATCA 82
Req, Get Device ID, seq 0x03
705 23.195024 172.18.243.59 172.18.240.41 IPMI/ATCA 98
Rsp, Get Device ID, seq 0x03
706 23.196501 172.18.240.41 172.18.243.59 IPMI/ATCA 82
Req, Get SDR Repository Info, seq 0x04
707 23.203028 172.18.243.59 172.18.240.41 IPMI/ATCA 94
Rsp, Get SDR Repository Info, seq 0x04
708 23.203276 172.18.240.41 172.18.243.59 IPMI/ATCA 82
Req, Get System GUID, seq 0x05
709 23.204426 172.18.243.59 172.18.240.41 IPMI/ATCA 98
Rsp, Get System GUID, seq 0x05
[... reading data...]
1070 26.328633 172.18.240.41 172.18.243.59 IPMI/ATCA 86
Req, Read FRU Data, seq 0x00
1071 26.335794 172.18.243.59 172.18.240.41 IPMI/ATCA 98
Rsp, Read FRU Data, seq 0x00
1072 26.342152 172.18.240.41 172.18.243.59 IPMI/ATCA 86
Req, Read FRU Data, seq 0x01
1073 26.348532 172.18.243.59 172.18.240.41 IPMI/ATCA 90
Rsp, Read FRU Data, seq 0x01
1074 26.362437 172.18.240.41 172.18.243.59 IPMI/ATCA 82
Req, Get FRU Inventory Area Info, seq 0x02
1075 26.368826 172.18.243.59 172.18.240.41 IPMI/ATCA 82
Rsp, Get FRU Inventory Area Info, seq 0x02, Requested Sensor, data, or record
not present
[...now back to first attempt???]
1076 26.634557 172.18.240.41 172.18.243.59 RMCP+ 86
Session ID 0x0, payload type: RAKP Message 3
1077 26.648660 172.18.243.59 172.18.240.41 RMCP+ 78
Session ID 0x0, payload type: RAKP Message 4
1080 32.654516 172.18.240.41 172.18.243.59 IPMI/ATCA 86
Req, Close Session, seq 0x03
1081 32.655545 172.18.243.59 172.18.240.41 IPMI/ATCA 82
Rsp, Close Session, seq 0x03
Other UDP ports:
[SOL channel poll...at end of above]
1078 27.982543 172.18.240.41 172.18.243.59 RMCP+ 106
Session ID 0x3006600, payload type: IPMI Message
1079 27.989680 172.18.243.59 172.18.240.41 RMCP+ 122
Session ID 0xa0a2a3a4, payload type: IPMI Message
[Now some apparent failures...]
1082 33.104841 172.18.240.41 172.18.243.59 IPMI/ATCA 65
Req, Get Channel Authentication Capabilities, seq 0x00
1083 33.128409 172.18.243.59 172.18.240.41 IPMI/ATCA 72
Rsp, Get Channel Authentication Capabilities, seq 0x00
1084 33.141633 172.18.240.41 172.18.243.59 RMCP+ 90
Session ID 0x0, payload type: RMCP+ Open Session Request
1085 33.143052 172.18.243.59 172.18.240.41 RMCP+ 94
Session ID 0x0, payload type: RMCP+ Open Session Response
1086 33.143600 172.18.240.41 172.18.243.59 RMCP+ 92
Session ID 0x0, payload type: RAKP Message 1
1087 33.190999 172.18.243.59 172.18.240.41 RMCP+ 118
Session ID 0x0, payload type: RAKP Message 2
[try again...]
1088 33.766187 172.18.240.41 172.18.243.59 IPMI/ATCA 65
Req, Get Channel Authentication Capabilities, seq 0x00
1089 33.788444 172.18.243.59 172.18.240.41 IPMI/ATCA 72
Rsp, Get Channel Authentication Capabilities, seq 0x00
1090 33.788799 172.18.240.41 172.18.243.59 RMCP+ 90
Session ID 0x0, payload type: RMCP+ Open Session Request
1091 33.789667 172.18.243.59 172.18.240.41 RMCP+ 66
Session ID 0x0, payload type: RMCP+ Open Session Response
1092 34.692467 172.18.240.41 172.18.243.59 RMCP+ 90
Session ID 0x0, payload type: RMCP+ Open Session Request
1093 34.693272 172.18.243.59 172.18.240.41 RMCP+ 66
Session ID 0x0, payload type: RMCP+ Open Session Response
1094 36.691968 172.18.240.41 172.18.243.59 RMCP+ 90
Session ID 0x0, payload type: RMCP+ Open Session Request
1095 36.692827 172.18.243.59 172.18.240.41 RMCP+ 66
Session ID 0x0, payload type: RMCP+ Open Session Response
1096 39.595609 172.18.240.41 172.18.243.59 RMCP+ 90
Session ID 0x0, payload type: RMCP+ Open Session Request
1097 39.596493 172.18.243.59 172.18.240.41 RMCP+ 66
Session ID 0x0, payload type: RMCP+ Open Session Response
1098 43.499700 172.18.240.41 172.18.243.59 RMCP+ 90
Session ID 0x0, payload type: RMCP+ Open Session Request
1099 43.500885 172.18.243.59 172.18.240.41 RMCP+ 66
Session ID 0x0, payload type: RMCP+ Open Session Response
[another SOL poll...]
1100 43.992449 172.18.240.41 172.18.243.59 RMCP+ 106
Session ID 0x3006600, payload type: IPMI Message
1101 43.999954 172.18.243.59 172.18.240.41 RMCP+ 122
Session ID 0xa0a2a3a4, payload type: IPMI Message
[Now it gets back on track...]
1102 48.723033 172.18.240.41 172.18.243.59 IPMI/ATCA 65
Req, Get Channel Authentication Capabilities, seq 0x00
1103 48.749519 172.18.243.59 172.18.240.41 IPMI/ATCA 72
Rsp, Get Channel Authentication Capabilities, seq 0x00
1104 48.755494 172.18.240.41 172.18.243.59 RMCP+ 90
Session ID 0x0, payload type: RMCP+ Open Session Request
1105 48.756323 172.18.243.59 172.18.240.41 RMCP+ 66
Session ID 0x0, payload type: RMCP+ Open Session Response
1106 49.447200 172.18.240.41 172.18.243.59 RMCP+ 90
Session ID 0x0, payload type: RMCP+ Open Session Request
1107 49.448500 172.18.243.59 172.18.240.41 RMCP+ 94
Session ID 0x0, payload type: RMCP+ Open Session Response
1108 49.448754 172.18.240.41 172.18.243.59 RMCP+ 92
Session ID 0x0, payload type: RAKP Message 1
1109 49.496820 172.18.243.59 172.18.240.41 RMCP+ 118
Session ID 0x0, payload type: RAKP Message 2
1110 49.497085 172.18.240.41 172.18.243.59 RMCP+ 86
Session ID 0x0, payload type: RAKP Message 3
1111 49.513293 172.18.243.59 172.18.240.41 RMCP+ 78
Session ID 0x0, payload type: RAKP Message 4
1112 49.515150 172.18.240.41 172.18.243.59 IPMI/ATCA 82
Req, Set Session Privilege Level, seq 0x02
1113 49.530868 172.18.243.59 172.18.240.41 IPMI/ATCA 82
Rsp, Set Session Privilege Level, seq 0x02
1114 49.565946 172.18.240.41 172.18.243.59 IPMI/ATCA 82
Req, Get Device ID, seq 0x03
1115 49.576630 172.18.243.59 172.18.240.41 IPMI/ATCA 98
Rsp, Get Device ID, seq 0x03
1116 49.591635 172.18.240.41 172.18.243.59 IPMI/ATCA 82
Req, Get SDR Repository Info, seq 0x04
1117 49.597564 172.18.243.59 172.18.240.41 IPMI/ATCA 94
Rsp, Get SDR Repository Info, seq 0x04
1118 49.620941 172.18.240.41 172.18.243.59 IPMI/ATCA 82
Req, Get System GUID, seq 0x05
1119 49.621971 172.18.243.59 172.18.240.41 IPMI/ATCA 98
Rsp, Get System GUID, seq 0x05
Port 59309 also went through similar oddities as port 44407. Went
through to RAKP2 the back restarted through to RAKP4 and then towards
the end did RAKP3/4 transitions again.
Almost like is is getting twice on bmc_handlers or sessions_waiting in
IPMI.pm. I notice bmc_handler is sometimes indexed by $ip and other
times by $host. I don't have time tonight to track those back to
ensure they are the same.
Hopefully these dumps can help. I'll look at things more tomorrow.
Stuart
--
I've never been lost; I was once bewildered for three days, but never lost!
-- Daniel Boone
------------------------------------------------------------------------------
Get your SQL database under version control now!
Version control is standard for application code, but databases havent
caught up. So what steps can you take to put your SQL databases under
version control? Why should you start doing it? Read more to find out.
http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk
_______________________________________________
xCAT-user mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/xcat-user
