Can you also double check ASPM is Disabled in lspci -vvv? This sounds like
what happens when we lose PCIe link due to some instability of the design.
Also please make sure the device is being cooled well enough.
like so (for aspm):
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
You may also want to try a different slot to see if that helps, or RMA the
adapter (but it could be a system issue).
-----Original Message-----
From: John Haechten [mailto:[email protected]]
Sent: Friday, November 18, 2011 3:23 PM
To: [email protected]
Subject: [E1000-devel] Issue with (Intel 82576 Gigabit - Quad Port Server
Adapter, ET2
I have run into an issue with an Intel 4 Port Ethernet Adapter card where ALL
the Ethernet ports on a particular card appear to get over-run with
Errors. All 4 Ports are hung up even though there was only 1 port doing active
data transfer.
In my configuration, I have 2 Ethernet ports on the motherboard, with the
following PCI-
Express cards:
- PCI-Express card with 10Gb x 2 Port (Intel 82599EB 10-Gigabit)
- PCI-Express card with 1Gb x 4 Port (Intel 82576 Gigabit - Quad Port
Server Adapter, Gigabit ET2 Intel no. E1G44ET2)
eth0,1 are 1Gb ports on the motherboard
Eth2,3 are 10Gb ports
Eth4,5,6,7 are 1Gb ports
Ethernet Ports 4,5,6,7 all show about the same symptoms at the same time, so it
is like the board has some issue or possibly the firmware on the board is
incompatible.
A sample of the errors:
RX packets:43568 errors:16922171142300
dropped:4230542785575 overruns:4230546819150
frame:16922171142300
TX packets:5079 errors:8461085571150 dropped:0 overruns:0
carrier:8461085571150
My client is able to mount across all the Ethernet ports. I create individual
mounts for each port.
I am able to do directory listings, etc. on the client and see the files, etc.
across each mount.
However, when I start transferring data over a single 1Gb port by copying the
data or using "dd", All 4 of the 1Gb Ethernet ports stop working and appears
to hang.
I am able to use the 2 x 10Gb Ports, so I think this is an issue with this
particular adapter card.
When I investigate the port, all the error counts are huge, even for ports that
were not actively involved in the data transfer.
However, this only appears to be true for the ports on that PCI-Express board,
no the Motherboard which has the same chipset.
What can cause this type of problem??
I am running CentOS 6.0
OS distribution: CentOS Linux release 6.0 (Final)
Linux kernel: 2.6.32-71.29.1.el6.x86_64
NOTE: I was running this configuration with CentOS5.5, Linux kernel: 2.6.35 and
did not have this issue. I then upgraded to CentOS 6.0.
I can transfer data across the 10Gb ports just fine. I was able to use the 2 x
10Gb ports and the 4 x 1Gb ports simultaneously with CentOS 5.5
Is there a possible compatibility issue with the 10Gb card using NAPI and the
1Gb card does not?
I have included the Ethernet Driver info for the 1Gb and the 10Gb drivers.
Also, I am trying to maximize my performance over the 1Gb and 10Gb ports, once
they are all working. Are there driver parameters that I need to reconfigure
to achieve this?
-bash-4.1# lspci | grep Ethernet
01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection
(rev 01)
01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection
(rev 01)
06:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
Network Connection (rev 01)
06:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
Network Connection (rev 01)
09:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection
(rev ff)
09:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection
(rev ff)
0a:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection
(rev ff)
0a:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection
(rev ff)
-bash-4.1# ifconfig
eth0 Link encap:Ethernet HWaddr 00:25:90:35:0B:30 inet
addr:192.168.40.233 Bcast:192.168.40.255
Mask:255.255.255.0
inet6 addr: fe80::225:90ff:fe35:b30/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:18796 errors:0 dropped:0 overruns:0 frame:0
TX packets:12948 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2598944 (2.4 MiB) TX bytes:3662092 (3.4 MiB)
eth2 Link encap:Ethernet HWaddr 00:1B:21:D4:63:DC
inet addr:192.168.31.110 Bcast:192.168.31.255
Mask:255.255.255.0
inet6 addr: fe80::21b:21ff:fed4:63dc/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:9760518 errors:8 dropped:0 overruns:0 frame:8
TX packets:27964383 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:47591717266 (44.3 GiB) TX bytes:36305129570 (33.8GiB)
eth3 Link encap:Ethernet HWaddr 00:1B:21:D4:63:DD
inet addr:192.168.32.110 Bcast:192.168.32.255
Mask:255.255.255.0
inet6 addr: fe80::21b:21ff:fed4:63dd/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:6269483 errors:29 dropped:0 overruns:0 frame:29
TX packets:14876350 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:32168238434 (29.9 GiB) TX bytes:18210194044 (16.9GiB)
eth4 Link encap:Ethernet HWaddr 00:1B:21:BA:F6:B8
inet addr:192.168.11.60 Bcast:192.168.11.255
Mask:255.255.255.0
inet6 addr: fe80::21b:21ff:feba:f6b8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:43568 errors:16922171142300
dropped:4230542785575 overruns:4230546819150
frame:16922171142300
TX packets:5079 errors:8461085571150 dropped:0 overruns:0
carrier:8461085571150
collisions:4230542785575 txqueuelen:1000
RX bytes:60579323 (57.7 MiB) TX bytes:354286 (345.9 KiB)
eth5 Link encap:Ethernet HWaddr 00:1B:21:BA:F6:B9
inet addr:192.168.12.60 Bcast:192.168.12.255
Mask:255.255.255.0
inet6 addr: fe80::21b:21ff:feba:f6b9/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3795 errors:16939351011480
dropped:4234837752870 overruns:4234841790540
frame:16939351011480
TX packets:227 errors:8469675505740 dropped:0 overruns:0
carrier:8469675505740
collisions:4234837752870 txqueuelen:1000
RX bytes:400313 (390.9 KiB) TX bytes:24890 (24.3 KiB)
eth6 Link encap:Ethernet HWaddr 00:1B:21:BA:F6:BA
inet addr:192.168.13.60 Bcast:192.168.13.255
Mask:255.255.255.0
inet6 addr: fe80::21b:21ff:feba:f6ba/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3787 errors:16939351011480
dropped:4234837752870 overruns:4234841790540
frame:16939351011480
TX packets:226 errors:8469675505740 dropped:0 overruns:0
carrier:8469675505740
collisions:4234837752870 txqueuelen:1000
RX bytes:399823 (390.4 KiB) TX bytes:24820 (24.2 KiB)
eth7 Link encap:Ethernet HWaddr 00:1B:21:BA:F6:BB
inet addr:192.168.14.60 Bcast:192.168.14.255
Mask:255.255.255.0
inet6 addr: fe80::21b:21ff:feba:f6bb/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3780 errors:16922171142300
dropped:4230542785575 overruns:4230546819150
frame:16922171142300
TX packets:226 errors:8461085571150 dropped:0 overruns:0
carrier:8461085571150
collisions:4230542785575 txqueuelen:1000
RX bytes:399306 (389.9 KiB) TX bytes:24820 (24.2 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:2140097 errors:0 dropped:0 overruns:0 frame:0
TX packets:2140097 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:565735588 (539.5 MiB) TX bytes:565735588 (539.5 MiB)
--------------------------------------------
1Gb Driver ModInfo for igb (eth0,eth1 on motherboard + 4 Port Card
with eth4,eth5,eth6.eth7)
-bash-4.1# modinfo igb
filename: /lib/modules/2.6.32-
71.29.1.el6.x86_64/kernel/drivers/net/igb/igb.ko
version: 3.2.10
license: GPL
description: Intel(R) Gigabit Ethernet Network Driver
author: Intel Corporation, <[email protected]>
srcversion: BAAD5948059DD794C854151
alias: pci:v00008086d000010D6sv*sd*bc*sc*i*
alias: pci:v00008086d000010A9sv*sd*bc*sc*i*
alias: pci:v00008086d000010A7sv*sd*bc*sc*i*
alias: pci:v00008086d000010E8sv*sd*bc*sc*i*
alias: pci:v00008086d00001526sv*sd*bc*sc*i*
alias: pci:v00008086d0000150Dsv*sd*bc*sc*i*
alias: pci:v00008086d000010E7sv*sd*bc*sc*i*
alias: pci:v00008086d000010E6sv*sd*bc*sc*i*
alias: pci:v00008086d00001518sv*sd*bc*sc*i*
alias: pci:v00008086d0000150Asv*sd*bc*sc*i*
alias: pci:v00008086d000010C9sv*sd*bc*sc*i*
alias: pci:v00008086d00000440sv*sd*bc*sc*i*
alias: pci:v00008086d0000043Csv*sd*bc*sc*i*
alias: pci:v00008086d0000043Asv*sd*bc*sc*i*
alias: pci:v00008086d00000438sv*sd*bc*sc*i*
alias: pci:v00008086d00001516sv*sd*bc*sc*i*
alias: pci:v00008086d00001511sv*sd*bc*sc*i*
alias: pci:v00008086d00001510sv*sd*bc*sc*i*
alias: pci:v00008086d00001527sv*sd*bc*sc*i*
alias: pci:v00008086d0000150Fsv*sd*bc*sc*i*
alias: pci:v00008086d0000150Esv*sd*bc*sc*i*
alias: pci:v00008086d00001524sv*sd*bc*sc*i*
alias: pci:v00008086d00001523sv*sd*bc*sc*i*
alias: pci:v00008086d00001522sv*sd*bc*sc*i*
alias: pci:v00008086d00001521sv*sd*bc*sc*i*
depends: dca
vermagic: 2.6.32-71.29.1.el6.x86_64 SMP mod_unload modversions
parm: InterruptThrottleRate:Maximum interrupts per second, per
vector, (max 100000), default 3=adaptive (array of int)
parm: IntMode:Change Interrupt Mode (0=Legacy, 1=MSI, 2=MSI-
X), default 2 (array of int)
parm: Node:set the starting node to allocate memory on, default -
1 (array of int)
parm: LLIPort:Low Latency Interrupt TCP Port (0-65535), default
0=off (array of int)
parm: LLIPush:Low Latency Interrupt on TCP Push flag (0,1),
default 0=off (array of int)
parm: LLISize:Low Latency Interrupt on Packet Size (0-1500),
default 0=off (array of int)
parm: RSS:Number of Receive-Side Scaling Descriptor Queues
(0-8), default 1=number of cpus (array of int)
parm: VMDQ:Number of Virtual Machine Device Queues: 0-1 =
disable, 2-8 enable, default 0 (array of int)
parm: max_vfs:Number of Virtual Functions: 0 = disable, 1-7
enable, default 0 (array of int)
parm: MDD:Malicious Driver Detection (0/1), default 1 = enabled.
Only available when max_vfs is greater than 0 (array of int)
parm: QueuePairs:Enable TX/RX queue pairs for interrupt
handling (0,1), default 1=on (array of int)
parm: EEE:Enable/disable on parts that support the feature (array
of int)
parm: DMAC:Disable or set latency for DMA Coalescing ((0=off,
1000-10000(msec), 250, 500 (usec)) (array of int)
parm: LRO:Large Receive Offload (0,1), default 0=off (array of int)
parm: debug:Debug level (0=none, ..., 16=all) (int)
--------------------------------------------
10Gb Driver ModInfo for ixgbe (eth2 and eth3)
-bash-4.1# modinfo ixgbe
filename: /lib/modules/2.6.32-
71.29.1.el6.x86_64/kernel/drivers/net/ixgbe/ixgbe.ko
version: 3.6.7-NAPI
license: GPL
description: Intel(R) 10 Gigabit PCI Express Network Driver
author: Intel Corporation, <[email protected]>
srcversion: 119ADBD2B16B0E40668FB9A
alias: pci:v00008086d0000154Fsv*sd*bc*sc*i*
alias: pci:v00008086d0000154Dsv*sd*bc*sc*i*
alias: pci:v00008086d00001528sv*sd*bc*sc*i*
alias: pci:v00008086d000010F8sv*sd*bc*sc*i*
alias: pci:v00008086d0000151Csv*sd*bc*sc*i*
alias: pci:v00008086d00001529sv*sd*bc*sc*i*
alias: pci:v00008086d0000152Asv*sd*bc*sc*i*
alias: pci:v00008086d000010F9sv*sd*bc*sc*i*
alias: pci:v00008086d00001514sv*sd*bc*sc*i*
alias: pci:v00008086d00001507sv*sd*bc*sc*i*
alias: pci:v00008086d000010FBsv*sd*bc*sc*i*
alias: pci:v00008086d00001517sv*sd*bc*sc*i*
alias: pci:v00008086d000010FCsv*sd*bc*sc*i*
alias: pci:v00008086d000010F7sv*sd*bc*sc*i*
alias: pci:v00008086d00001508sv*sd*bc*sc*i*
alias: pci:v00008086d000010DBsv*sd*bc*sc*i*
alias: pci:v00008086d000010F4sv*sd*bc*sc*i*
alias: pci:v00008086d000010E1sv*sd*bc*sc*i*
alias: pci:v00008086d000010F1sv*sd*bc*sc*i*
alias: pci:v00008086d000010ECsv*sd*bc*sc*i*
alias: pci:v00008086d000010DDsv*sd*bc*sc*i*
alias: pci:v00008086d0000150Bsv*sd*bc*sc*i*
alias: pci:v00008086d000010C8sv*sd*bc*sc*i*
alias: pci:v00008086d000010C7sv*sd*bc*sc*i*
alias: pci:v00008086d000010C6sv*sd*bc*sc*i*
alias: pci:v00008086d000010B6sv*sd*bc*sc*i*
depends: dca
vermagic: 2.6.32-71.29.1.el6.x86_64 SMP mod_unload modversions
parm: InterruptType:Change Interrupt Mode (0=Legacy, 1=MSI,
2=MSI-X), default IntMode (deprecated) (array of int)
parm: IntMode:Change Interrupt Mode (0=Legacy, 1=MSI, 2=MSI-
X), default 2 (array of int)
parm: Node:set the starting node to allocate memory on, default -
1 (array of int)
parm: MQ:Disable or enable Multiple Queues, default 1 (array of
int)
parm: DCA:Disable or enable Direct Cache Access, 0=disabled,
1=descriptor only, 2=descriptor and data (array of int)
parm: RSS:Number of Receive-Side Scaling Descriptor Queues,
default 1=number of cpus (array of int)
parm: VMDQ:Number of Virtual Machine Device Queues: 0/1 =
disable, 2-16 enable (default=8) (array of int)
parm: max_vfs:Number of Virtual Functions: 0 = disable (default),
1 = default settings, 2-63 = enable this many VFs (array of int)
parm: L2LBen:L2 Loopback Enable: 0 = disable, 1 = enable
(default) (array of int)
parm: InterruptThrottleRate:Maximum interrupts per second, per
vector, (0,1,956-488281), default 1 (array of int)
parm: LLIPort:Low Latency Interrupt TCP Port (0-65535) (array of
int)
parm: LLIPush:Low Latency Interrupt on TCP Push flag (0,1)
(array of int)
parm: LLISize:Low Latency Interrupt on Packet Size (0-1500)
(array of int)
parm: LLIEType:Low Latency Interrupt Ethernet Protocol Type
(array of int)
parm: LLIVLANP:Low Latency Interrupt on VLAN priority threshold
(array of int)
parm: RxBufferMode:0=1 descriptor per packet,
1=use packet split, multiple descriptors per jumbo frame
2 (default)=use 1buf mode for 1500 mtu, packet split for
jumbo (array of int)
parm: FdirMode:Flow Director filtering modes:
0 = Filtering off
1 = Signature Hashing filters (SW ATR)
2 = Perfect Filters (array of int)
parm: FdirPballoc:Flow Director packet buffer allocation level:
1 = 8k hash filters or 2k perfect filters
2 = 16k hash filters or 4k perfect filters
3 = 32k hash filters or 8k perfect filters (array of
int)
parm: AtrSampleRate:Software ATR Tx packet sample rate (array
of int)
parm: FCoE:Disable or enable FCoE Offload, default 1 (array of
int)
parm: LRO:Large Receive Offload (0,1), default 1 = on (array of int)
------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure
contains a definitive record of customers, application performance,
security threats, fraudulent activity, and more. Splunk takes this
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
_______________________________________________
E1000-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit
http://communities.intel.com/community/wired
