As Adam guessed, I do infact have Intel 82546GB-based NICs on this
Supermicro board. With using the newest Intel 7.3.20 driver, the IPMI
traffic does get through the but the OS also seems to be getting the
traffic as well. Because it doesn't have any daemons listening to that
traffic, it's sending back huge amounts of ICMP destination unreachable
packets back to the source.
The majority of my IPMI machines are Broadcom based which use a
dedicated MAC address and this solution is very reliable there. I'm
curious, what new Intel chips use dedicated MAC addresses for IPMI
communications? I'm getting the impression that I need to avoid the use
of 82546-based boards.
Back to the original issue here: the Supermicro-provided "ipmiview" and
"ipmicli" tools seem to work reliably with this Intel-based setup but
the ipmitool-1.8.8.90 version does not. Maybe the issue of the ICMP
errors are screwing up ipmitool? It doesn't look like it from the debug
below as it looks more like duplicate packets possibly from a bad Intel
driver? Anyway, here is the "-vvvv" output that Tom requested.
Thoughts anyone?
--David
--
dolly-p8sci:/usr/src/archive/ipmi/ipmitool-1.8.8.90/src# ipmitool -vvvv
-U ADMIN -P ADMIN -I lanplus -H 10.159.4.7 mc info
IPMI LAN host 10.159.4.7 port 623
>> Sending IPMI command payload
>> netfn : 0x06
>> command : 0x38
>> data : 0x8e 0x04
BUILDING A v1.5 COMMAND
>> IPMI Request Session Header
>> Authtype : NONE
>> Sequence : 0x00000000
>> Session ID : 0x00000000
>> IPMI Request Message Header
>> Rs Addr : 20
>> NetFn : 06
>> Rs LUN : 0
>> Rq Addr : 81
>> Rq Seq : 00
>> Rq Lun : 0
>> Command : 38
Invalid RMCP class: 0
<< IPMI Response Session Header
<< Authtype : NONE
<< Payload type : IPMI (0)
<< Session ID : 0x00000000
<< Sequence : 0x00000000
<< IPMI Msg/Payload Length : 16
<< IPMI Response Message Header
<< Rq Addr : 81
<< NetFn : 07
<< Rq LUN : 0
<< Rs Addr : 20
<< Rq Seq : 00
<< Rs Lun : 0
<< Command : 38
<< Compl Code : 0x00
IPMI Request Match found
>> SENDING AN OPEN SESSION REQUEST
Invalid RMCP class: 3
<<OPEN SESSION RESPONSE
<< Message tag : 0x00
<< RMCP+ status : no errors
<< Maximum privilege level : admin
<< Console Session ID : 0xa0a2a3a4
<< BMC Session ID : 0x02007301
<< Negotiated authenticatin algorithm : hmac_sha1
<< Negotiated integrity algorithm : hmac_sha1_96
<< Negotiated encryption algorithm : aes_cbc_128
>> Console generated random number (16 bytes)
c9 66 39 84 9e 6c b7 e7 50 33 0c 3e 15 ac de ab
>> SENDING A RAKP 1 MESSAGE
Error: Received an Unexpected Open Session Response
<<RAKP 2 MESSAGE
<< Message tag : 0x00
<< RMCP+ status : no errors
<< Console Session ID : 0xa0a2a3a4
<< BMC random number : 0x69dca4eba216228877ebf7642ea169ae
<< BMC GUID : 0x20000e43027f78080000000000000000
<< Key exchange auth code [sha1] :
0x9f3271209e50463212b00d68e261f086f3487557
bmc_rand (16 bytes)
69 dc a4 eb a2 16 22 88 77 eb f7 64 2e a1 69 ae
>> rakp2 mac input buffer (63 bytes)
a4 a3 a2 a0 01 73 00 02 c9 66 39 84 9e 6c b7 e7
50 33 0c 3e 15 ac de ab 69 dc a4 eb a2 16 22 88
77 eb f7 64 2e a1 69 ae 20 00 0e 43 02 7f 78 08
00 00 00 00 00 00 00 00 14 05 41 44 4d 49 4e
>> rakp2 mac key (20 bytes)
41 44 4d 49 4e 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00
>> rakp2 mac as computed by the remote console (20 bytes)
9f 32 71 20 9e 50 46 32 12 b0 0d 68 e2 61 f0 86
f3 48 75 57
>> rakp3 mac input buffer (27 bytes)
69 dc a4 eb a2 16 22 88 77 eb f7 64 2e a1 69 ae
a4 a3 a2 a0 14 05 41 44 4d 49 4e
>> rakp3 mac key (20 bytes)
41 44 4d 49 4e 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00
generated rakp3 mac (20 bytes)
27 8d fb da f0 ad ee 39 44 be f8 0b 5b 31 2c f1
0b c9 bc b0
session integrity key input (39 bytes)
c9 66 39 84 9e 6c b7 e7 50 33 0c 3e 15 ac de ab
69 dc a4 eb a2 16 22 88 77 eb f7 64 2e a1 69 ae
14 05 41 44 4d 49 4e
Generated session integrity key (20 bytes)
72 7a 26 9e 2b 64 98 c3 bc e7 1b 82 f9 95 4d be
69 58 0e 19
Generated K1 (20 bytes)
34 3a 18 26 12 74 9c 50 f2 10 18 52 f2 a8 55 ba
32 c6 41 fa
Generated K2 (20 bytes)
af 39 15 8e 26 0e 11 dd 75 bc 7d 2b e1 df 92 7e
e3 88 a8 83
>> SENDING A RAKP 3 MESSAGE
Error: Received an Unexpected RAKP 2 message
<<RAKP 4 MESSAGE
<< Message tag : 0x00
<< RMCP+ status : no errors
<< Console Session ID : 0xa0a2a3a4
<< Key exchange auth code [sha1] : 0x4ce33c9e7d73158773f29526
>> rakp4 mac input buffer (36 bytes)
c9 66 39 84 9e 6c b7 e7 50 33 0c 3e 15 ac de ab
01 73 00 02 20 00 0e 43 02 7f 78 08 00 00 00 00
00 00 00 00
>> rakp4 mac key (sik) (20 bytes)
72 7a 26 9e 2b 64 98 c3 bc e7 1b 82 f9 95 4d be
69 58 0e 19
>> rakp4 mac as computed by the BMC (20 bytes)
4c e3 3c 9e 7d 73 15 87 73 f2 95 26 2e a1 69 ae
20 00 0e 43
>> rakp4 mac as computed by the remote console (20 bytes)
4c e3 3c 9e 7d 73 15 87 73 f2 95 26 d0 83 22 7c
03 ac 56 be
IPMIv2 / RMCP+ SESSION OPENED SUCCESSFULLY
>> Sending IPMI command payload
>> netfn : 0x06
>> command : 0x3b
>> data : 0x04
BUILDING A v2 COMMAND
>> Initialization vector (16 bytes)
3b c4 b8 35 fa 57 0d d5 ff 70 b8 2d cf 79 4d 1e
authcode input (48 bytes)
06 c0 01 73 00 02 03 00 00 00 20 00 3b c4 b8 35
fa 57 0d d5 ff 70 b8 2d cf 79 4d 1e 13 1e 8d ba
24 f9 6e 0d a2 4f 00 19 84 33 73 58 ff ff 02 07
authcode output (12 bytes)
42 6f 45 7c 6a 19 b8 90 fd 39 c2 1b
packet session id 0x0 does not match active session 0xa0a2a3a4
ipmitool: lanplus.c:1153: read_session_data_v2x: Assertion `0' failed.
Aborted
dolly-p8sci:/usr/src/archive/ipmi/ipmitool-1.8.8.90/src# ipmitool -vvvv
-U ADMIN -P ADMIN -I lanplus -H 10.159.4.7 mc info
IPMI LAN host 10.159.4.7 port 623
>> Sending IPMI command payload
>> netfn : 0x06
>> command : 0x38
>> data : 0x8e 0x04
BUILDING A v1.5 COMMAND
>> IPMI Request Session Header
>> Authtype : NONE
>> Sequence : 0x00000000
>> Session ID : 0x00000000
>> IPMI Request Message Header
>> Rs Addr : 20
>> NetFn : 06
>> Rs LUN : 0
>> Rq Addr : 81
>> Rq Seq : 00
>> Rq Lun : 0
>> Command : 38
Invalid RMCP class: 0
<< IPMI Response Session Header
<< Authtype : NONE
<< Payload type : IPMI (0)
<< Session ID : 0x00000000
<< Sequence : 0x00000000
<< IPMI Msg/Payload Length : 16
<< IPMI Response Message Header
<< Rq Addr : 81
<< NetFn : 07
<< Rq LUN : 0
<< Rs Addr : 20
<< Rq Seq : 00
<< Rs Lun : 0
<< Command : 38
<< Compl Code : 0x00
IPMI Request Match found
>> SENDING AN OPEN SESSION REQUEST
Invalid RMCP class: 3
<<OPEN SESSION RESPONSE
<< Message tag : 0x00
<< RMCP+ status : no errors
<< Maximum privilege level : admin
<< Console Session ID : 0xa0a2a3a4
<< BMC Session ID : 0x02007902
<< Negotiated authenticatin algorithm : hmac_sha1
<< Negotiated integrity algorithm : hmac_sha1_96
<< Negotiated encryption algorithm : aes_cbc_128
>> Console generated random number (16 bytes)
41 a9 bb 1f 1c c7 a5 db 88 9f ea 88 6e fe 0d d9
>> SENDING A RAKP 1 MESSAGE
Error: Received an Unexpected Open Session Response
<<RAKP 2 MESSAGE
<< Message tag : 0x00
<< RMCP+ status : no errors
<< Console Session ID : 0xa0a2a3a4
<< BMC random number : 0xfb6f7b6c087c886f5cd0dcc6ad212d1a
<< BMC GUID : 0x20000e43027f78080000000000000000
<< Key exchange auth code [sha1] :
0x1dcfa822e07d4e5d5e8637d0b4f548d7a944fd86
bmc_rand (16 bytes)
fb 6f 7b 6c 08 7c 88 6f 5c d0 dc c6 ad 21 2d 1a
>> rakp2 mac input buffer (63 bytes)
a4 a3 a2 a0 02 79 00 02 41 a9 bb 1f 1c c7 a5 db
88 9f ea 88 6e fe 0d d9 fb 6f 7b 6c 08 7c 88 6f
5c d0 dc c6 ad 21 2d 1a 20 00 0e 43 02 7f 78 08
00 00 00 00 00 00 00 00 14 05 41 44 4d 49 4e
>> rakp2 mac key (20 bytes)
41 44 4d 49 4e 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00
>> rakp2 mac as computed by the remote console (20 bytes)
1d cf a8 22 e0 7d 4e 5d 5e 86 37 d0 b4 f5 48 d7
a9 44 fd 86
>> rakp3 mac input buffer (27 bytes)
fb 6f 7b 6c 08 7c 88 6f 5c d0 dc c6 ad 21 2d 1a
a4 a3 a2 a0 14 05 41 44 4d 49 4e
>> rakp3 mac key (20 bytes)
41 44 4d 49 4e 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00
generated rakp3 mac (20 bytes)
ac ca 69 4f f2 3d 32 25 8c 5e d8 3e a4 94 59 3d
8e 14 21 dc
session integrity key input (39 bytes)
41 a9 bb 1f 1c c7 a5 db 88 9f ea 88 6e fe 0d d9
fb 6f 7b 6c 08 7c 88 6f 5c d0 dc c6 ad 21 2d 1a
14 05 41 44 4d 49 4e
Generated session integrity key (20 bytes)
72 2b 65 62 bc ba ab 0e 3d 1e 66 00 5e 5c be 85
2c a7 07 b3
Generated K1 (20 bytes)
8d 82 68 70 86 94 8b 9f b5 2d 71 ab e8 9d 37 70
c2 45 d6 8b
Generated K2 (20 bytes)
59 2f 38 4b 67 f8 2a cd f6 88 44 43 4a d3 d7 e0
df d6 63 ad
>> SENDING A RAKP 3 MESSAGE
<<RAKP 4 MESSAGE
<< Message tag : 0x00
<< RMCP+ status : no errors
<< Console Session ID : 0xa0a2a3a4
<< Key exchange auth code [sha1] : 0xeaf6907c6868254055dc10bd
>> rakp4 mac input buffer (36 bytes)
41 a9 bb 1f 1c c7 a5 db 88 9f ea 88 6e fe 0d d9
02 79 00 02 20 00 0e 43 02 7f 78 08 00 00 00 00
00 00 00 00
>> rakp4 mac key (sik) (20 bytes)
72 2b 65 62 bc ba ab 0e 3d 1e 66 00 5e 5c be 85
2c a7 07 b3
>> rakp4 mac as computed by the BMC (20 bytes)
ea f6 90 7c 68 68 25 40 55 dc 10 bd ad 21 2d 1a
20 00 0e 43
>> rakp4 mac as computed by the remote console (20 bytes)
ea f6 90 7c 68 68 25 40 55 dc 10 bd 55 13 17 fc
92 54 dd 7f
IPMIv2 / RMCP+ SESSION OPENED SUCCESSFULLY
>> Sending IPMI command payload
>> netfn : 0x06
>> command : 0x3b
>> data : 0x04
BUILDING A v2 COMMAND
>> Initialization vector (16 bytes)
84 58 42 5e c3 2c 6d fd 07 da 76 6e 2c 72 e8 9b
authcode input (48 bytes)
06 c0 02 79 00 02 03 00 00 00 20 00 84 58 42 5e
c3 2c 6d fd 07 da 76 6e 2c 72 e8 9b 37 bc 8e a2
f7 86 72 ae 48 d1 0d 8c fc 3f a1 a5 ff ff 02 07
authcode output (12 bytes)
98 f4 85 54 bd 7a b7 74 a7 ac 49 26
Validating authcode
K1 (20 bytes)
8d 82 68 70 86 94 8b 9f b5 2d 71 ab e8 9d 37 70
c2 45 d6 8b
Authcode Input Data (48 bytes)
06 c0 a4 a3 a2 a0 01 00 00 00 20 00 fb 6f 7b 6c
08 7c 88 6f 5c d0 dc c6 ad 21 2d 1a 88 df 6d fb
c6 18 cf 95 c6 11 14 32 74 25 f4 7d ff ff 02 07
Generated authcode (12 bytes)
3d b8 7f 3a f1 94 c2 6d 15 7b 45 c8
Expected authcode (12 bytes)
3d b8 7f 3a f1 94 c2 6d 15 7b 45 c8
<< IPMI Response Session Header
<< Authtype : Unknown (0x6)
<< Payload type : IPMI (0)
<< Session ID : 0xa0a2a3a4
<< Sequence : 0x00000001
<< IPMI Msg/Payload Length : 32
<< IPMI Response Message Header
<< Rq Addr : 81
<< NetFn : 07
<< Rq LUN : 0
<< Rs Addr : 20
<< Rq Seq : 01
<< Rs Lun : 0
<< Command : 3b
<< Compl Code : 0x00
IPMI Request Match found
set_session_privlvl (1 bytes)
04
Set Session Privilege Level to ADMINISTRATOR
>> Sending IPMI command payload
>> netfn : 0x06
>> command : 0x01
>> data :
BUILDING A v2 COMMAND
>> Initialization vector (16 bytes)
13 a0 33 67 8f 28 be 5d f9 57 5c 3c 8b f2 03 05
authcode input (48 bytes)
06 c0 02 79 00 02 04 00 00 00 20 00 13 a0 33 67
8f 28 be 5d f9 57 5c 3c 8b f2 03 05 3b d2 c4 58
f8 eb 6d 47 78 d8 29 3d c2 11 00 22 ff ff 02 07
authcode output (12 bytes)
7b 36 12 30 9b 02 41 fc 04 27 29 c6
Validating authcode
K1 (20 bytes)
8d 82 68 70 86 94 8b 9f b5 2d 71 ab e8 9d 37 70
c2 45 d6 8b
Authcode Input Data (64 bytes)
06 c0 a4 a3 a2 a0 02 00 00 00 30 00 fb 6f 7b 6c
08 7c 88 6f 5c d0 dc c6 ad 21 2d 1a e9 52 be b8
8d 66 6b a0 54 32 be 1e 21 db 39 61 d0 c1 69 7e
30 a9 ab d7 50 01 99 cf 76 9b 9f 1b ff ff 02 07
Generated authcode (12 bytes)
9d 4b 23 1c f6 10 2c 62 9c fd 0a 75
Expected authcode (12 bytes)
9d 4b 23 1c f6 10 2c 62 9c fd 0a 75
<< IPMI Response Session Header
<< Authtype : Unknown (0x6)
<< Payload type : IPMI (0)
<< Session ID : 0xa0a2a3a4
<< Sequence : 0x00000002
<< IPMI Msg/Payload Length : 48
<< IPMI Response Message Header
<< Rq Addr : 81
<< NetFn : 07
<< Rq LUN : 0
<< Rs Addr : 20
<< Rq Seq : 02
<< Rs Lun : 0
<< Command : 01
<< Compl Code : 0x00
IPMI Request Match found
Device ID : 32
Device Revision : 0
Firmware Revision : 2.4
IPMI Version : 2.0
Manufacturer ID : 5593
Manufacturer Name : Unknown (0x15d9)
Product ID : 4404 (0x1134)
Device Available : yes
Provides Device SDRs : no
Additional Device Support :
Sensor Device
SDR Repository Device
SEL Device
FRU Inventory Device
IPMB Event Receiver
IPMB Event Generator
Bridge
Aux Firmware Rev Info :
0x00
0x00
0x00
0x00
>> Sending IPMI command payload
>> netfn : 0x06
>> command : 0x3c
>> data : 0x02 0x79 0x00 0x02
BUILDING A v2 COMMAND
>> Initialization vector (16 bytes)
0b 02 ec 31 27 43 26 d4 04 6f f9 6c e0 5d ab 34
authcode input (48 bytes)
06 c0 02 79 00 02 05 00 00 00 20 00 0b 02 ec 31
27 43 26 d4 04 6f f9 6c e0 5d ab 34 07 d5 55 e2
62 5a 9e 3d 24 56 a2 ae 9d 87 06 12 ff ff 02 07
authcode output (12 bytes)
d6 ed fa a2 50 7b 0b a0 6f 96 ff 54
Validating authcode
K1 (20 bytes)
8d 82 68 70 86 94 8b 9f b5 2d 71 ab e8 9d 37 70
c2 45 d6 8b
Authcode Input Data (48 bytes)
06 c0 a4 a3 a2 a0 03 00 00 00 20 00 fb 6f 7b 6c
08 7c 88 6f 5c d0 dc c6 ad 21 2d 1a e3 32 90 0b
a6 04 53 61 8c 45 43 e9 86 97 79 e0 ff ff 02 07
Generated authcode (12 bytes)
72 10 95 01 5d fc b3 3d 99 a5 c0 40
Expected authcode (12 bytes)
72 10 95 01 5d fc b3 3d 99 a5 c0 40
<< IPMI Response Session Header
<< Authtype : Unknown (0x6)
<< Payload type : IPMI (0)
<< Session ID : 0xa0a2a3a4
<< Sequence : 0x00000003
<< IPMI Msg/Payload Length : 32
<< IPMI Response Message Header
<< Rq Addr : 81
<< NetFn : 07
<< Rq LUN : 0
<< Rs Addr : 20
<< Rq Seq : 03
<< Rs Lun : 0
<< Command : 3c
<< Compl Code : 0x00
IPMI Request Match found
Closed Session 02007902
--
OK, I had to respond to that misconception wrt e1000 & BMCs. :-)
The real reason that a separate MAC (sometimes a separate NIC) was
provided for BMC traffic on some new Intel motherboards was that there
are more features being put onto the management NIC than just port 623
can handle (HTTP, SMTP, KVM, TELNET, etc.), so sharing the MAC was not
feasible for those additional features.
Do note that some newer NIC chips do require e1000 driver updates, as is
normal when new devices are introduced.
Andy
________________________________
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of
Thomas Adam Nabarro
Sent: Thursday, January 25, 2007 5:17 PM
To: ipmitool-devel@lists.sourceforge.net
Subject: RE: Re: [Ipmitool-devel] Lead developer still busy, extra
project
In response to David:
I'm not sure if this will offer any help but the new intel bmc's have
dedicated MAC's for ipmi traffic and seperate MAC's for OS. Therefore it
might be that the e1000 driver is not providing a multiplexor type
operation correctly for discriminating between OS and IPMI traffic.
Intel provided a dedicated MAC for the BMC for the reason that IPMI
communication was not always reliable if MAC was shared, and problems
were encountered such as you are seeing + bandwidth issues.
I hope this helps, but as supermicro tools: ipmiview and ipmicli work
reliably, i am unsure if the issues you are experiencing are related to
the information i have given you. It may be that these tools provide
different authentication steps to ipmitool, which enable them to work
more reliably with the e1000 driver.
If you give more detail (-vvvv), the data may give more insight.
Thanks
Tom Nabarro
------------------------------
Message: 4
Date: Thu, 25 Jan 2007 10:08:07 -0800
From: "David A. Ranch" <[EMAIL PROTECTED]>
Subject: Re: [Ipmitool-devel] Lead developer still busy, extra project
admin appointed
To: ipmitool-devel@lists.sourceforge.net
Message-ID: <[EMAIL PROTECTED]>
Content-Type: text/plain; charset=ISO-8859-2; format=flowed
I'm seeing a issue here with both 1.8.8 and this
ipmitool/ipmitool-1.8.8.90 build (on CENTOS 4.4). When communicating
to
a SuperMicro BMC connected via Intel MACs (which share the same
Ethernet
MAC and IP address with the OS itself), the IPMI communications are not
reliable. 70% of the time things work, 30% the time, it seems the
communication is reported to be garbled. This is using the
non-standard
Centos 3.3 Intel e1000 driver version 7.3.20 as the stock version
doesn't work at all (all communications fail with that e1000 driver).
What's strange to me is that:
1. If the machine (Supermicro H8QC8 (4x Opteron 244) is not running
Linux (say it's in DOS, waiting the BIOS configuration area, etc.),
ipmitool works fine. It's only when Linux is loaded do these issues
appear. But..
2. If I use Supermicro's "ipmiview" or "ipmicli" tool for Windows or
Linux, everything works fine regardless of the loaded OS on the
machine:
ftp://ftp.supermicro.com/CDR-0010_2.03_for_IPMI_Server_Managment/Rev2.10
_Beta/IPMI_Solution/
Thoughts?
Ps. I've notice that wireshark v. IPMI decodes are rather incomplete.
Does anyone have a recommended way to work with that community to get
this tightened up?
--David
------------------------------------------------------------------------
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
------------------------------------------------------------------------
_______________________________________________
Ipmitool-devel mailing list
Ipmitool-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ipmitool-devel
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Ipmitool-devel mailing list
Ipmitool-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ipmitool-devel
