Donald, thank you for answer. On Thu, Jan 7, 2016 at 3:11 AM, Skidmore, Donald C < donald.c.skidm...@intel.com> wrote: > > Sorry our hardware doesn't support LX4 modules, which is what I'm assuming > you mean by WDM SFP+. The unsupported module parameter is only used for > modules that we recognize but haven't validated. This way we at least know > the module won't hurt the hardware even if we can't vouch for their > performance. Attempting to use any unknown SFP+ module that was plugged > in (assuming zero init) could potently harm the HW. >
Well, I'm not sure this is LX4 module. According to vendor this module is: - 10GBASE-BX at 10.3125 Gb/s - 10GBASE-BX at 9.953 Gb/s - 1000 Base-LX Ethernet So this is not standard module for sure. Still vendor suggests that this module is supposed work with most hardware and it really works with cisco/juniper hardware. The question is, are there any hints that may help to make module work with intel card? -- Peter. > -----Original Message----- > > From: Peter Volkov [mailto:peter.vol...@gmail.com] > > Sent: Tuesday, January 05, 2016 8:54 PM > > To: e1000-devel@lists.sourceforge.net > > Subject: [E1000-devel] ixgbe: link flapping on two directly connected > servers > > with 82599ES adapter > > > > Good time. > > > > We experince link flapping while connecting two supermicro servers with > > 82599ES adapters: > > > > [Wed Jan 6 07:10:14 2016] ixgbe 0000:81:00.1 enp129s0f1: NIC Link is > Down > > [Wed Jan 6 07:10:15 2016] ixgbe 0000:81:00.1 enp129s0f1: NIC Link is Up > 10 > > Gbps, Flow Control: RX/TX [Wed Jan 6 07:10:15 2016] ixgbe 0000:81:00.1 > > enp129s0f1: NIC Link is Down [Wed Jan 6 07:10:16 2016] ixgbe > 0000:81:00.1 > > enp129s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [Wed Jan 6 > > 07:10:16 2016] ixgbe 0000:81:00.1 enp129s0f1: NIC Link is Down [Wed Jan > 6 > > 07:10:16 2016] ixgbe 0000:81:00.1 enp129s0f1: NIC Link is Up 10 Gbps, > Flow > > Control: RX/TX [Wed Jan 6 07:10:16 2016] ixgbe 0000:81:00.1 enp129s0f1: > > NIC Link is Down [Wed Jan 6 07:10:16 2016] ixgbe 0000:81:00.1 > enp129s0f1: > > NIC Link is Up 10 Gbps, Flow Control: RX/TX [Wed Jan 6 07:10:17 2016] > > ixgbe 0000:81:00.1 enp129s0f1: NIC Link is Down [Wed Jan 6 07:10:17 > 2016] > > ixgbe 0000:81:00.1 enp129s0f1: NIC Link is Up 10 Gbps, Flow Control: > RX/TX > > > > So we are trying to connect two supermicro servers with E10G42BTDA > > X520-DA2 controller > > in each server: > > 81:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit > SFI/SFP+ > > Network Connection (rev 01) > > > > In one controller there is following SFP module: > > > > # ethtool --module-info enp129s0f1 > > Identifier : 0x03 (SFP) > > Extended identifier : 0x04 (GBIC/SFP defined by > > 2-wire interface ID) > > Connector : 0x07 (LC) > > Transceiver codes : 0x20 0x00 0x00 0x00 0x00 0x00 > > 0x00 0x00 > > Transceiver type : 10G Ethernet: 10G Base-LR > > Encoding : 0x06 (64B/66B) > > BR, Nominal : 10300MBd > > Rate identifier : 0x00 (unspecified) > > Length (SMF,km) : 20km > > Length (SMF) : 20000m > > Length (50um) : 0m > > Length (62.5um) : 0m > > Length (Copper) : 0m > > Length (OM3) : 0m > > Laser wavelength : 1330nm > > Vendor name : GIGALINK > > Vendor OUI : 00:90:65 > > Vendor PN : GL-OT-ST12LC1-13 > > Vendor rev : A > > Optical diagnostics support : Yes > > Laser bias current : 37.020 mA > > Laser output power : 0.9654 mW / -0.15 dBm > > Receiver signal average optical power : 0.4904 mW / -3.09 dBm > > Module temperature : 44.23 degrees C / 111.61 > > degrees F > > Module voltage : 3.2836 V > > Alarm/warning flags implemented : Yes > > Laser bias current high alarm : Off > > Laser bias current low alarm : Off > > Laser bias current high warning : Off > > Laser bias current low warning : Off > > Laser output power high alarm : Off > > Laser output power low alarm : Off > > Laser output power high warning : Off > > Laser output power low warning : Off > > Module temperature high alarm : Off > > Module temperature low alarm : Off > > Module temperature high warning : Off > > Module temperature low warning : Off > > Module voltage high alarm : Off > > Module voltage low alarm : Off > > Module voltage high warning : Off > > Module voltage low warning : Off > > Laser rx power high alarm : Off > > Laser rx power low alarm : Off > > Laser rx power high warning : Off > > Laser rx power low warning : Off > > Laser bias current high alarm threshold : 85.000 mA > > Laser bias current low alarm threshold : 10.000 mA > > Laser bias current high warning threshold : 80.000 mA Laser bias current > low > > warning threshold : 12.000 mA > > Laser output power high alarm threshold : 3.1623 mW / 5.00 dBm > > Laser output power low alarm threshold : 0.3162 mW / -5.00 dBm > > Laser output power high warning threshold : 2.5119 mW / 4.00 dBm Laser > > output power low warning threshold : 0.3981 mW / -4.00 dBm > > Module temperature high alarm threshold : 85.00 degrees C / 185.00 > > degrees F > > Module temperature low alarm threshold : -10.00 degrees C / 14.00 > > degrees F > > Module temperature high warning threshold : 80.00 degrees C / 176.00 > > degrees F Module temperature low warning threshold : -5.00 degrees C / > > 23.00 degrees F > > Module voltage high alarm threshold : 3.7000 V > > Module voltage low alarm threshold : 2.9000 V > > Module voltage high warning threshold : 3.6000 V > > Module voltage low warning threshold : 3.0000 V > > Laser rx power high alarm threshold : 1.0000 mW / 0.00 dBm > > Laser rx power low alarm threshold : 0.0200 mW / -16.99 dBm > > Laser rx power high warning threshold : 0.7943 mW / -1.00 dBm > > Laser rx power low warning threshold : 0.0251 mW / -16.00 dBm > > > > In another controller: > > > > # ethtool --module-info enp129s0f1 > > Identifier : 0x03 (SFP) > > Extended identifier : 0x04 (GBIC/SFP defined by > > 2-wire interface ID) > > Connector : 0x07 (LC) > > Transceiver codes : 0x20 0x00 0x00 0x00 0x00 0x00 > > 0x00 0x00 > > Transceiver type : 10G Ethernet: 10G Base-LR > > Encoding : 0x06 (64B/66B) > > BR, Nominal : 10300MBd > > Rate identifier : 0x00 (unspecified) > > Length (SMF,km) : 20km > > Length (SMF) : 20000m > > Length (50um) : 0m > > Length (62.5um) : 0m > > Length (Copper) : 0m > > Length (OM3) : 0m > > Laser wavelength : 1270nm > > Vendor name : GIGALINK > > Vendor OUI : 00:90:65 > > Vendor PN : GL-OT-ST12LC1-12 > > Vendor rev : A > > Option values : 0x00 0x1a > > Option : RX_LOS implemented > > Option : TX_FAULT implemented > > Option : TX_DISABLE implemented > > BR margin, max : 0% > > BR margin, min : 0% > > Vendor SN : G201511300616 > > Date code : 151118 > > Optical diagnostics support : Yes > > Laser bias current : 35.120 mA > > Laser output power : 0.7842 mW / -1.06 dBm > > Receiver signal average optical power : 0.5695 mW / -2.45 dBm > > Module temperature : 45.68 degrees C / 114.22 > > degrees F > > Module voltage : 3.2440 V > > Alarm/warning flags implemented : Yes > > Laser bias current high alarm : Off > > Laser bias current low alarm : Off > > Laser bias current high warning : Off > > Laser bias current low warning : Off > > Laser output power high alarm : Off > > Laser output power low alarm : Off > > Laser output power high warning : Off > > Laser output power low warning : Off > > Module temperature high alarm : Off > > Module temperature low alarm : Off > > Module temperature high warning : Off > > Module temperature low warning : Off > > Module voltage high alarm : Off > > Module voltage low alarm : Off > > Module voltage high warning : Off > > Module voltage low warning : Off > > Laser rx power high alarm : Off > > Laser rx power low alarm : Off > > Laser rx power high warning : Off > > Laser rx power low warning : Off > > Laser bias current high alarm threshold : 85.000 mA > > Laser bias current low alarm threshold : 10.000 mA > > Laser bias current high warning threshold : 80.000 mA Laser bias current > low > > warning threshold : 12.000 mA > > Laser output power high alarm threshold : 3.1623 mW / 5.00 dBm > > Laser output power low alarm threshold : 0.3162 mW / -5.00 dBm > > Laser output power high warning threshold : 2.5119 mW / 4.00 dBm Laser > > output power low warning threshold : 0.3981 mW / -4.00 dBm > > Module temperature high alarm threshold : 85.00 degrees C / 185.00 > > degrees F > > Module temperature low alarm threshold : -10.00 degrees C / 14.00 > > degrees F > > Module temperature high warning threshold : 80.00 degrees C / 176.00 > > degrees F Module temperature low warning threshold : -5.00 degrees C / > > 23.00 degrees F > > Module voltage high alarm threshold : 3.7000 V > > Module voltage low alarm threshold : 2.9000 V > > Module voltage high warning threshold : 3.6000 V > > Module voltage low warning threshold : 3.0000 V > > Laser rx power high alarm threshold : 1.0000 mW / 0.00 dBm > > Laser rx power low alarm threshold : 0.0200 mW / -16.99 dBm > > Laser rx power high warning threshold : 0.7943 mW / -1.00 dBm > > Laser rx power low warning threshold : 0.0251 mW / -16.00 dBm > > > > > > We've checked both SFP+ modules and patchcord in other hardware and link > > was working fine there. As for module version we've tried both 4.1.5 and > > 4.3.13 and we experience problem with both versions. I've built v4.3.13 > > module with printk enabled, and I found following messages in dmesg: > > > > [Wed Jan 6 06:04:02 2016] ixgbe 0000:81:00.1 enp129s0f1: NIC Link is Up > 10 > > Gbps, Flow Control: RX/TX [Wed Jan 6 06:04:02 2016] > > ixgbe_get_media_type_82599 [Wed Jan 6 06:04:02 2016] > > ixgbe_check_mac_link_generic [Wed Jan 6 06:04:02 2016] > > ixgbe_get_media_type_82599 [Wed Jan 6 06:04:02 2016] ixgbe > > 0000:81:00.1 enp129s0f1: NIC Link is Down [Wed Jan 6 06:04:02 2016] > > ixgbe_check_mac_link_genericixgbe_fc_enable_genericixgbe_fc_autonegixg > > be_check_mac_link_generic > > [Wed Jan 6 06:04:02 2016] ixgbe 0000:81:00.1 enp129s0f1: NIC Link is Up > 10 > > Gbps, Flow Control: RX/TX [Wed Jan 6 06:04:02 2016] > > ixgbe_get_media_type_82599 [Wed Jan 6 06:04:02 2016] > > ixgbe_check_mac_link_generic [Wed Jan 6 06:04:02 2016] > > ixgbe_get_media_type_82599 [Wed Jan 6 06:04:02 2016] ixgbe > > 0000:81:00.1 enp129s0f1: NIC Link is Down > > > > Does this helps to understand reasons for such behaviour? > > > > What makes me wonder. In Documentation/networking/ixgbe.txt I found > > following information > > > > *82599-BASED ADAPTERS* > > > > *NOTES: If your 82599-based Intel(R) Network Adapter came with Intel > > optics, or* *is an Intel(R) Ethernet Server Adapter X520-2, then it only > > supports Intel* *optics and/or the direct attach cables listed below.* > > > > *When 82599-based SFP+ devices are connected back to back, they should > > be set to* *the same Speed setting via ethtool. Results may vary if you > mix > > speed > > settings.* > > *82598-based adapters support all passive direct attach cables that > comply* > > *with SFF-8431 v4.1 and SFF-8472 v10.4 specifications. Active direct > attach* > > *cables are not supported.* > > > > Yet any attempt to set speed fails: > > # ethtool -s enp129s0f1 speed 10000 > > Cannot set new settings: Invalid argument > > not setting speed > > > > Why is that? That said I found suggestion to set advertising mode: > > > > ethtool -s enp129s0f1 advertise 0x1000 > > > > and this worked, but link flapping continues. > > > > Yes, I realize that these modules are unsupported and to make them > working > > I had to load module with allow_unsupported_sfp=1,1 but the main problem > > we have is that I failed to find any intel suggested WDM SFP+ module. We > > need to use only single fiber cable since in our final configuration we > need to > > connect both servers to our provider and they highly recomend WDM > > connections. If these modules are not going to work does there exist any > > WDM module that will? > > > > # ethtool -i enp129s0f1 > > driver: ixgbe > > version: 4.1.5 > > firmware-version: 0x61c10001 > > bus-info: 0000:81:00.1 > > supports-statistics: yes > > supports-test: yes > > supports-eeprom-access: yes > > supports-register-dump: yes > > supports-priv-flags: no > > > > # ethtool enp129s0f1 > > Settings for enp129s0f1: > > Supported ports: [ FIBRE ] > > Supported link modes: 10000baseT/Full > > Supported pause frame use: No > > Supports auto-negotiation: No > > Advertised link modes: 10000baseT/Full > > Advertised pause frame use: Symmetric > > Advertised auto-negotiation: No > > Speed: 10000Mb/s > > Duplex: Full > > Port: FIBRE > > PHYAD: 0 > > Transceiver: external > > Auto-negotiation: off > > Supports Wake-on: d > > Wake-on: d > > Current message level: 0x00000007 (7) > > drv probe link > > Link detected: yes > > > > We've struggle with this problem for few days already so any suggestions > are > > more then wellcome. > > > > > > TIA, > > -- > > Peter. >
------------------------------------------------------------------------------
_______________________________________________ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
