> On Aug 10, 2016, at 10:12 PM, Ian Collins <[email protected]> wrote: > > On 11/08/16 1:08 pm, Robert Mustacchi wrote: >> On 8/10/16 18:01 , Robert Mustacchi wrote: >>> Okay, thanks for providing that Ian. I'm trying to think through what >>> might cause this case and how we might better handle this. I'm reviewing >>> the PRM but will probably need some time on this one. >>> >>> One question I have is on this particular box are those on-board ports >>> shared with the BMC? And are you using the dedicated BMC ports? I'm just >>> wondering if, like the issue Dale mentioned with the X550, because we >>> have a shared management port, the link was up and we never really reset >>> it properly in the driver to run at full speed vs. it having been >>> programmed for other, more limited speeds by the BMC. > > No, the BMC has it's own port. >> Aha, the X540 has LPLU (Low Power Link Up) mode. So, I bet that we have >> this in use on your X540 that's onboard, but not on the PCIe card. >> >> Ian, if I have some time over the next week I can put together a custom >> platform to test and verify that this is the case and with a potential >> fix. Are these machines in a place where using a test platform would be >> acceptable? > > Yes, they are warming my office!
Yeah, this is the same problem I ran into with the embedded X550 on the Xeon-D (X550EM) ... all the X540 variants I tested with were PCIe cards. The problem stems from the card initializing at power-on and going into LPLU mode immediately. When the kernel gets around to booting and the ixgbe driver loads, the ixgbe driver (traditionally) has always presumed that the card will, upon the earlier power-on initialization, sync up at the fastest speed it can determine... which for non-LPLU situations is usually 10Gb (peer and cabling permitting, of course)... so ixgbe never bothered forcing it to re-neg the link. This is why when you use dladm to diddle the autoneg speed settings later on after boot, it'll come up at 10Gb. It surprises me that the X540 has a LPLU in a on-board form... I can't recollect that being mentioned in the docs. It just means we need to add X540 to the list of MACs to force a on-start autonegotiation in ixgbe. /dale
signature.asc
Description: Message signed with OpenPGP using GPGMail
------------------------------------------- smartos-discuss Archives: https://www.listbox.com/member/archive/184463/=now RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00 Modify Your Subscription: https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb Powered by Listbox: http://www.listbox.com
