Okay, I am at my wits end here with these NICs.

We have a total of six X10SDV-TNL4F motherboards (2 x i350 & 2 x X557)
connected to a QuantaMesh T3048-LY9 running CumulusLinux 2.5.5 (the
QuantaMesh is using Trident II chips if that matters).

I am having a complete devil of a time getting the X557 network interfaces
to connect at 10Gb. It seems to be completely random.

Out of the six servers currently here is what happened on the latest
rebuild (Ubuntu 14.04 LTS with 3.0.19.0-43. kernel and 4.2.5 driver):

Server 1: eth0 is connected at 10Gb, eth1 is connected at 1Gb
Server 2: eth0 is not connected at all, eth1 is connected at 1Gb
Server 3: eth0 is connected at 10Gb, eth1 is connected at 10Gb
Server 4: eth0 is connected at 10Gb, eth1 is connected at 1Gb
Server 5: eth0 is connected at 10Gb, eth1 is connected at 10Gb
Server 6: eth0 is connected at 10Gb, eth1 is connected at 10Gb

Server 1 and 2 are configured identically (X10SDV-TLN4F, 32GB of RAM, RAID1
SSDs), Servers 3-6 are identical (X10SDV-8C-TLN4F, 64GB RAM, 16GB SATADOM)

I have tried the following (in various combinations):

* Multiple driver versions 4.1.5, 4.2.5, 4.3.9, 4.3.13 (just released
earlier today) none of which seems to make a difference.
* Updated the BIOS on the motherboard
* setting: pre-up ethtool -s eth0[1] advertise 0x1000
* Multiple cables (Cat 6 rated up to 550Mhz - cables themselves are only 2
meters long - Don't have a 10Gb cable tester to certify them though)
* Turned off jumbo frames

On some servers, when I set the pre-up advertise command and reboot, the
interfaces never come back online.

As an added data point, we have three servers with two X540-AT2 interfaces
in them connected to the same switch. Bonded interfaces, they all work
flawlessly on the same switch, they link up at 10Gb without an issue using
the same cables (using the in tree driver).

I am struggling to find any commonality between any of the servers to even
start to debug this problem.

As an additional aside, if I bounce an interface (ifdown eth0; ifup eth0)
the interface never reconnects, I have to reboot the server for it to
reconnect.

Help me, please.

Tom Walsh
------------------------------------------------------------------------------
_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit 
http://communities.intel.com/community/wired

Reply via email to