Thanks for the feedback, 2.7.7 and 2.8 will contain the workaround and I don't foresee a reason for us to ever revive that check.
From: Sten Wolf <[email protected]> To: xCAT Users Mailing list <[email protected]>, Date: 01/31/2013 08:24 PM Subject: Re: [xcat-user] ipmi commands not working if used as "rXXX" comands, but work as ipmitool-xcat commands Thanks, That solved the issue. I'm trying to raise the issue with Supermicro as well, hopefully they'll fix their IPMI. On 29/01//2013 18:14, Jarrod B Johnson wrote: Ok, looked through the capture. The BMC violates the IPMI spec by not correctly setting the target address field. (e.g. you can see in wireshark that the protocol dissector blows up on the packet by saying 'Unkown command' and claiming it says 'BROADCAST', this is all because the byte at 0x38 is '00' and it should have been '0x81'). xCAT blows up for the same reason, the BMC is required to correctly address responses and xCAT recently started caring about that. xCAT actually blows up less than the dissector, the dissector actually gets confused about the correct offsets for the rest of the packet. The good news is the reason for using multiple rqaddr values (why our routing code had to actually check the value) was removed and thus we can retire that check as well and tolerate the spec violation. Can you see if the following change allows it to work again? --- a/xCAT-server/lib/perl/xCAT/IPMI.pm +++ b/xCAT-server/lib/perl/xCAT/IPMI.pm @@ -865,7 +865,7 @@ sub parse_ipmi_payload { my @payload = @_; #for now, just trash the headers, this has been validated to death anyway #except seqlun, that one is important - unless ($payload[4] == $self->{seqlun} and $payload[0] == $self->{rqaddr} and $payload[1]>>2 == $self->{expectednetfn} and $payload[5] == $self->{expecte + unless ($payload[4] == $self->{seqlun} and $payload[1]>>2 == $self->{expectednetfn} and $payload[5] == $self->{expectedcmd} ) { #both sequence number and arqaddr must match, because we are using rqaddr to extend the sequence number #print "Successfully didn't get confused by stale response ".$payload[4]." and ".($self->{seqlun}-4)."\n"; #hexdump(@payload); Sten Wolf ---01/29/2013 08:54:46 AM---Attached wireshark captures of the "conversation" stream. tux401ipmi.cap has the issue. From: Sten Wolf <[email protected]> To: xCAT Users Mailing list <[email protected]>, Date: 01/29/2013 08:54 AM Subject: Re: [xcat-user] ipmi commands not working if used as "rXXX" comands, but work as ipmitool-xcat commands Attached wireshark captures of the "conversation" stream. tux401ipmi.cap has the issue. Interestingly, when attaching the node to an older xcat server ( running on centos 6.2 ) the result is correct. The capture for the same node, from the older xcat server is attached as tux401ipmi-external.cap trying perl -e 'use Digest::SHA1; use Digest::HMAC_SHA1;' didn't return anything. trying ipmitool -I lanplus -C 2 <etc...> returned correct result, same as without -C 2 trying perl -e 'use Crypt::Rijndael; use Crypt::CBC;' returned error: Can't locate Crypt/Rijndael.pm in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at -e line 1. BEGIN failed--compilation aborted at -e line 1. trying /opt/xcat/lib/perl/xCAT/IPMI.pm. After 'Digest::HMAC_HA1-> import()' there is a '1;'. Change that to '0;' : the result was the same as with '1' - timeout. should I have done anything else after modifying IPMI.pm? from comparing the cap files, it seems the new server doesn't send the rmcp+ open session request, sending the same "get channel" packet over and over... any idea what can be done? On 28/01//2013 16:56, Jarrod B Johnson wrote: It may be interesting to review a libpcap capture at the time of the attempt. I can probably more specifically ascertain the issue. I'll make a few speculative guesses as what could theoretically be the problem. -BMC is demanding IPMI 2, but xCAT's ipmi2support is failing. Try: perl -e 'use Digest::SHA1; use Digest::HMAC_SHA1;'. If that works, then xCAT should be doing IPMI 2 by default if the BMC advertises it. I might should make those hard rpm dependencies, since they are in-distro for almost everyone. -xCAT is doing IPMI2 and BMC requires IPMI2, but it more specifically is requiring cipher suite 3 and xCAT's prereqs for that are missing. To test this, try ipmitool -I lanplus -C 2 <etc...>' and test for xCAT Cipher suite 3 by doing: perl -e 'use Crypt::Rijndael; use Crypt::CBC;' -Service processor can do IPMI 2, but does not support mutual authentication (protection against impersonating a BMC). ipmitool works because it doesn't bother to authenticate the BMC at all (which actually means rcons is hijackable so long as we use ipmitool). The most secure workaround to try in such a case is to search for 'ipmi2support' in /opt/xcat/lib/perl/xCAT/IPMI.pm. After 'Digest::HMAC_HA1-> import()' there is a '1;'. Change that to '0;' to force IPMI 1.5 operation. *If* they don't support IPMI 2 mutual authentication, then 1.5 would be more secure. Note that doesn't mean that any functionality is lost, we still issue IPMI 2 version commands when support detected even if we use IPMI 1.5 protocol to convey them. If you were interested in explicitly testing that last case, you'd have to disable the check for matching authtype, the return statement commented as 'not thinking about packets that do not match our preferred auth type' would be that. I probably should add a site entry to disable IPMI 2 support. I could also more specifically relax the mutual authentication check, but I'd much rather the relatively few manufacturers that can't pass that fix their firmware since it's a much better security picture. Sten Wolf ---01/27/2013 08:07:50 PM---Hi, I'm encountering a strange issue with some SuperMicro machines, using xcat 2.7.6. All nodes are From: Sten Wolf <[email protected]> To: [email protected], Date: 01/27/2013 08:07 PM Subject: [xcat-user] ipmi commands not working if used as "rXXX" comands, but work as ipmitool-xcat commands Hi, I'm encountering a strange issue with some SuperMicro machines, using xcat 2.7.6. All nodes are centos 6.3 (both MN and compute nodes). After configuring nodehm, ipmi tables the rpower/rinv/rinstall/rvitals commands do not work. They simply time out. using the same credentials with ipmitool-xcat returns correct results. # ipmitool-xcat -I lanplus -H ipmi05 -U ADMIN -P ADMIN power status Chassis Power is on while # rpower n05 status n05: Error: timeout Firewall and selinux both disabled. lsdef n05 confirms it's bmc is ipmi05. using xcatdebug -f enable -d enable also confirms the command is trying to connect to the ipmi address, with the correct username and password, but getting no reply. The only possible issue I could see is 'ipmiversion' => '1.5', I haven't tried ipmitool -I lan yet, but I wonder if this setting is user configurable in xcat? BTW - nodeset works correctly, if I power cycle the nodes manually, or with ipmitool, they install without further issues. ------------------------------------------------------------------------------ Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnnow-d2d _______________________________________________ xCAT-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/xcat-user [attachment "tux401ipmi-external.cap" deleted by Jarrod B Johnson/Raleigh/IBM] [attachment "tux401ipmi.cap" deleted by Jarrod B Johnson/Raleigh/IBM] ------------------------------------------------------------------------------ Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnnow-d2d _______________________________________________ xCAT-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/xcat-user ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_jan _______________________________________________ xCAT-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/xcat-user
<<inline: graycol.gif>>
------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_jan
_______________________________________________ xCAT-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/xcat-user
