I looked in the BIOS config and found that the ASPM setting was already disabled. Just to be sure of where this is located: Advanced - Chipset Control - North Bridge
I found the Active-State Power Mgmt Disabled for the North Bridge. This was the only location that I found reference to anything in the BIOS settings resembling ASPM. Is this correct? John -----Original Message----- From: Brandeburg, Jesse [mailto:[email protected]] Sent: Monday, November 21, 2011 5:03 PM To: John Haechten; Wyborny, Carolyn; [email protected] Subject: RE: [E1000-devel] Issue with (Intel 82576 Gigabit - Quad Port Server Adapter, ET2 Try disabling ASPM in the pcie configuration stuff in the BIOS, first. -----Original Message----- From: John Haechten [mailto:[email protected]] Sent: Monday, November 21, 2011 11:31 AM To: Wyborny, Carolyn; Brandeburg, Jesse; [email protected] Subject: RE: [E1000-devel] Issue with (Intel 82576 Gigabit - Quad Port Server Adapter, ET2 My config has ASPM enabled, so How do you disable ASPM? Also, CommClk+ vs. CommClk- LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- Vs. > Jesse's output: >LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- -----Original Message----- From: Wyborny, Carolyn [mailto:[email protected]] Sent: Monday, November 21, 2011 12:12 PM To: Brandeburg, Jesse; John Haechten; [email protected] Subject: RE: [E1000-devel] Issue with (Intel 82576 Gigabit - Quad Port Server Adapter, ET2 The output looks to be from after the problem has happened. The bus info after the quad 82576 card is all unretrievable in the lspci outputs. Can you also reboot the system and provide new lspci outputs so we can look at the setup and the ASPM status. As Jesse suggested, there appears to be something specific to the slot. We'll start with ASPM disabling. Is there another slot you can put quad port card into if that doesn't solve the problem? Thanks, Carolyn >-----Original Message----- >From: Brandeburg, Jesse [mailto:[email protected]] >Sent: Monday, November 21, 2011 9:20 AM >To: John Haechten; [email protected] >Subject: Re: [E1000-devel] Issue with (Intel 82576 Gigabit - Quad Port >Server Adapter, ET2 > >Can you also double check ASPM is Disabled in lspci -vvv? This sounds >like what happens when we lose PCIe link due to some instability of the >design. Also please make sure the device is being cooled well enough. > >like so (for aspm): > LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- >CommClk- > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > >You may also want to try a different slot to see if that helps, or RMA >the adapter (but it could be a system issue). > >-----Original Message----- >From: John Haechten [mailto:[email protected]] >Sent: Friday, November 18, 2011 3:23 PM >To: [email protected] >Subject: [E1000-devel] Issue with (Intel 82576 Gigabit - Quad Port >Server Adapter, ET2 > >I have run into an issue with an Intel 4 Port Ethernet Adapter card >where ALL the Ethernet ports on a particular card appear to get over-run >with >Errors. All 4 Ports are hung up even though there was only 1 port doing >active data transfer. >In my configuration, I have 2 Ethernet ports on the motherboard, with >the following PCI- >Express cards: > - PCI-Express card with 10Gb x 2 Port (Intel 82599EB 10-Gigabit) > - PCI-Express card with 1Gb x 4 Port (Intel 82576 Gigabit - Quad >Port >Server Adapter, Gigabit ET2 Intel no. E1G44ET2) >eth0,1 are 1Gb ports on the motherboard >Eth2,3 are 10Gb ports >Eth4,5,6,7 are 1Gb ports >Ethernet Ports 4,5,6,7 all show about the same symptoms at the same >time, so it is like the board has some issue or possibly the firmware on >the board is incompatible. >A sample of the errors: > RX packets:43568 errors:16922171142300 >dropped:4230542785575 overruns:4230546819150 >frame:16922171142300 > > TX packets:5079 errors:8461085571150 dropped:0 overruns:0 >carrier:8461085571150 > >My client is able to mount across all the Ethernet ports. I create >individual mounts for each port. >I am able to do directory listings, etc. on the client and see the >files, etc. across each mount. >However, when I start transferring data over a single 1Gb port by >copying the data or using "dd", All 4 of the 1Gb Ethernet ports stop >working and appears to hang. >I am able to use the 2 x 10Gb Ports, so I think this is an issue with >this particular adapter card. > >When I investigate the port, all the error counts are huge, even for >ports that were not actively involved in the data transfer. > >However, this only appears to be true for the ports on that PCI-Express >board, no the Motherboard which has the same chipset. > >What can cause this type of problem?? > >I am running CentOS 6.0 > >OS distribution: CentOS Linux release 6.0 (Final) > >Linux kernel: 2.6.32-71.29.1.el6.x86_64 > >NOTE: I was running this configuration with CentOS5.5, Linux kernel: >2.6.35 and did not have this issue. I then upgraded to CentOS 6.0. >I can transfer data across the 10Gb ports just fine. I was able to use >the 2 x 10Gb ports and the 4 x 1Gb ports simultaneously with CentOS 5.5 >Is there a possible compatibility issue with the 10Gb card using NAPI >and the 1Gb card does not? >I have included the Ethernet Driver info for the 1Gb and the 10Gb >drivers. >Also, I am trying to maximize my performance over the 1Gb and 10Gb >ports, once they are all working. Are there driver parameters that I >need to reconfigure to achieve this? > >-bash-4.1# lspci | grep Ethernet > >01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network >Connection (rev 01) > >01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network >Connection (rev 01) > >06:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit >SFI/SFP+ Network Connection (rev 01) > >06:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit >SFI/SFP+ Network Connection (rev 01) > >09:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network >Connection (rev ff) > >09:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network >Connection (rev ff) > >0a:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network >Connection (rev ff) > >0a:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network >Connection (rev ff) > >-bash-4.1# ifconfig >eth0 Link encap:Ethernet HWaddr 00:25:90:35:0B:30 inet >addr:192.168.40.233 Bcast:192.168.40.255 >Mask:255.255.255.0 > inet6 addr: fe80::225:90ff:fe35:b30/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:18796 errors:0 dropped:0 overruns:0 frame:0 > TX packets:12948 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:2598944 (2.4 MiB) TX bytes:3662092 (3.4 MiB) > >eth2 Link encap:Ethernet HWaddr 00:1B:21:D4:63:DC > inet addr:192.168.31.110 Bcast:192.168.31.255 >Mask:255.255.255.0 > inet6 addr: fe80::21b:21ff:fed4:63dc/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:9760518 errors:8 dropped:0 overruns:0 frame:8 > TX packets:27964383 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:47591717266 (44.3 GiB) TX bytes:36305129570 >(33.8GiB) > >eth3 Link encap:Ethernet HWaddr 00:1B:21:D4:63:DD > inet addr:192.168.32.110 Bcast:192.168.32.255 >Mask:255.255.255.0 > inet6 addr: fe80::21b:21ff:fed4:63dd/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:6269483 errors:29 dropped:0 overruns:0 frame:29 > TX packets:14876350 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:32168238434 (29.9 GiB) TX bytes:18210194044 >(16.9GiB) > >eth4 Link encap:Ethernet HWaddr 00:1B:21:BA:F6:B8 > inet addr:192.168.11.60 Bcast:192.168.11.255 >Mask:255.255.255.0 > inet6 addr: fe80::21b:21ff:feba:f6b8/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:43568 errors:16922171142300 >dropped:4230542785575 overruns:4230546819150 >frame:16922171142300 > TX packets:5079 errors:8461085571150 dropped:0 overruns:0 >carrier:8461085571150 > collisions:4230542785575 txqueuelen:1000 > RX bytes:60579323 (57.7 MiB) TX bytes:354286 (345.9 KiB) > >eth5 Link encap:Ethernet HWaddr 00:1B:21:BA:F6:B9 > inet addr:192.168.12.60 Bcast:192.168.12.255 >Mask:255.255.255.0 > inet6 addr: fe80::21b:21ff:feba:f6b9/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:3795 errors:16939351011480 >dropped:4234837752870 overruns:4234841790540 >frame:16939351011480 > TX packets:227 errors:8469675505740 dropped:0 overruns:0 >carrier:8469675505740 > collisions:4234837752870 txqueuelen:1000 > RX bytes:400313 (390.9 KiB) TX bytes:24890 (24.3 KiB) > >eth6 Link encap:Ethernet HWaddr 00:1B:21:BA:F6:BA > inet addr:192.168.13.60 Bcast:192.168.13.255 >Mask:255.255.255.0 > inet6 addr: fe80::21b:21ff:feba:f6ba/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:3787 errors:16939351011480 >dropped:4234837752870 overruns:4234841790540 >frame:16939351011480 > TX packets:226 errors:8469675505740 dropped:0 overruns:0 >carrier:8469675505740 > collisions:4234837752870 txqueuelen:1000 > RX bytes:399823 (390.4 KiB) TX bytes:24820 (24.2 KiB) > >eth7 Link encap:Ethernet HWaddr 00:1B:21:BA:F6:BB > inet addr:192.168.14.60 Bcast:192.168.14.255 >Mask:255.255.255.0 > inet6 addr: fe80::21b:21ff:feba:f6bb/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:3780 errors:16922171142300 >dropped:4230542785575 overruns:4230546819150 >frame:16922171142300 > TX packets:226 errors:8461085571150 dropped:0 overruns:0 >carrier:8461085571150 > collisions:4230542785575 txqueuelen:1000 > RX bytes:399306 (389.9 KiB) TX bytes:24820 (24.2 KiB) > >lo Link encap:Local Loopback > inet addr:127.0.0.1 Mask:255.0.0.0 > inet6 addr: ::1/128 Scope:Host > UP LOOPBACK RUNNING MTU:16436 Metric:1 > RX packets:2140097 errors:0 dropped:0 overruns:0 frame:0 > TX packets:2140097 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:0 > RX bytes:565735588 (539.5 MiB) TX bytes:565735588 (539.5 MiB) >-------------------------------------------- > >1Gb Driver ModInfo for igb (eth0,eth1 on motherboard + 4 Port Card >with eth4,eth5,eth6.eth7) > >-bash-4.1# modinfo igb >filename: /lib/modules/2.6.32- >71.29.1.el6.x86_64/kernel/drivers/net/igb/igb.ko >version: 3.2.10 >license: GPL >description: Intel(R) Gigabit Ethernet Network Driver >author: Intel Corporation, <[email protected]> >srcversion: BAAD5948059DD794C854151 >alias: pci:v00008086d000010D6sv*sd*bc*sc*i* >alias: pci:v00008086d000010A9sv*sd*bc*sc*i* >alias: pci:v00008086d000010A7sv*sd*bc*sc*i* >alias: pci:v00008086d000010E8sv*sd*bc*sc*i* >alias: pci:v00008086d00001526sv*sd*bc*sc*i* >alias: pci:v00008086d0000150Dsv*sd*bc*sc*i* >alias: pci:v00008086d000010E7sv*sd*bc*sc*i* >alias: pci:v00008086d000010E6sv*sd*bc*sc*i* >alias: pci:v00008086d00001518sv*sd*bc*sc*i* >alias: pci:v00008086d0000150Asv*sd*bc*sc*i* >alias: pci:v00008086d000010C9sv*sd*bc*sc*i* >alias: pci:v00008086d00000440sv*sd*bc*sc*i* >alias: pci:v00008086d0000043Csv*sd*bc*sc*i* >alias: pci:v00008086d0000043Asv*sd*bc*sc*i* >alias: pci:v00008086d00000438sv*sd*bc*sc*i* >alias: pci:v00008086d00001516sv*sd*bc*sc*i* >alias: pci:v00008086d00001511sv*sd*bc*sc*i* >alias: pci:v00008086d00001510sv*sd*bc*sc*i* >alias: pci:v00008086d00001527sv*sd*bc*sc*i* >alias: pci:v00008086d0000150Fsv*sd*bc*sc*i* >alias: pci:v00008086d0000150Esv*sd*bc*sc*i* >alias: pci:v00008086d00001524sv*sd*bc*sc*i* >alias: pci:v00008086d00001523sv*sd*bc*sc*i* >alias: pci:v00008086d00001522sv*sd*bc*sc*i* >alias: pci:v00008086d00001521sv*sd*bc*sc*i* >depends: dca >vermagic: 2.6.32-71.29.1.el6.x86_64 SMP mod_unload modversions >parm: InterruptThrottleRate:Maximum interrupts per second, per >vector, (max 100000), default 3=adaptive (array of int) >parm: IntMode:Change Interrupt Mode (0=Legacy, 1=MSI, 2=MSI- >X), default 2 (array of int) >parm: Node:set the starting node to allocate memory on, >default - >1 (array of int) >parm: LLIPort:Low Latency Interrupt TCP Port (0-65535), >default >0=off (array of int) >parm: LLIPush:Low Latency Interrupt on TCP Push flag (0,1), >default 0=off (array of int) >parm: LLISize:Low Latency Interrupt on Packet Size (0-1500), >default 0=off (array of int) >parm: RSS:Number of Receive-Side Scaling Descriptor Queues >(0-8), default 1=number of cpus (array of int) >parm: VMDQ:Number of Virtual Machine Device Queues: 0-1 = >disable, 2-8 enable, default 0 (array of int) >parm: max_vfs:Number of Virtual Functions: 0 = disable, 1-7 >enable, default 0 (array of int) >parm: MDD:Malicious Driver Detection (0/1), default 1 = >enabled. >Only available when max_vfs is greater than 0 (array of int) >parm: QueuePairs:Enable TX/RX queue pairs for interrupt >handling (0,1), default 1=on (array of int) >parm: EEE:Enable/disable on parts that support the feature >(array >of int) >parm: DMAC:Disable or set latency for DMA Coalescing ((0=off, >1000-10000(msec), 250, 500 (usec)) (array of int) >parm: LRO:Large Receive Offload (0,1), default 0=off (array of >int) >parm: debug:Debug level (0=none, ..., 16=all) (int) > >-------------------------------------------- > >10Gb Driver ModInfo for ixgbe (eth2 and eth3) > >-bash-4.1# modinfo ixgbe > >filename: /lib/modules/2.6.32- >71.29.1.el6.x86_64/kernel/drivers/net/ixgbe/ixgbe.ko >version: 3.6.7-NAPI >license: GPL >description: Intel(R) 10 Gigabit PCI Express Network Driver >author: Intel Corporation, <[email protected]> >srcversion: 119ADBD2B16B0E40668FB9A >alias: pci:v00008086d0000154Fsv*sd*bc*sc*i* >alias: pci:v00008086d0000154Dsv*sd*bc*sc*i* >alias: pci:v00008086d00001528sv*sd*bc*sc*i* >alias: pci:v00008086d000010F8sv*sd*bc*sc*i* >alias: pci:v00008086d0000151Csv*sd*bc*sc*i* >alias: pci:v00008086d00001529sv*sd*bc*sc*i* >alias: pci:v00008086d0000152Asv*sd*bc*sc*i* >alias: pci:v00008086d000010F9sv*sd*bc*sc*i* >alias: pci:v00008086d00001514sv*sd*bc*sc*i* >alias: pci:v00008086d00001507sv*sd*bc*sc*i* >alias: pci:v00008086d000010FBsv*sd*bc*sc*i* >alias: pci:v00008086d00001517sv*sd*bc*sc*i* >alias: pci:v00008086d000010FCsv*sd*bc*sc*i* >alias: pci:v00008086d000010F7sv*sd*bc*sc*i* >alias: pci:v00008086d00001508sv*sd*bc*sc*i* >alias: pci:v00008086d000010DBsv*sd*bc*sc*i* >alias: pci:v00008086d000010F4sv*sd*bc*sc*i* >alias: pci:v00008086d000010E1sv*sd*bc*sc*i* >alias: pci:v00008086d000010F1sv*sd*bc*sc*i* >alias: pci:v00008086d000010ECsv*sd*bc*sc*i* >alias: pci:v00008086d000010DDsv*sd*bc*sc*i* >alias: pci:v00008086d0000150Bsv*sd*bc*sc*i* >alias: pci:v00008086d000010C8sv*sd*bc*sc*i* >alias: pci:v00008086d000010C7sv*sd*bc*sc*i* >alias: pci:v00008086d000010C6sv*sd*bc*sc*i* >alias: pci:v00008086d000010B6sv*sd*bc*sc*i* >depends: dca >vermagic: 2.6.32-71.29.1.el6.x86_64 SMP mod_unload modversions >parm: InterruptType:Change Interrupt Mode (0=Legacy, 1=MSI, >2=MSI-X), default IntMode (deprecated) (array of int) >parm: IntMode:Change Interrupt Mode (0=Legacy, 1=MSI, 2=MSI- >X), default 2 (array of int) >parm: Node:set the starting node to allocate memory on, >default - >1 (array of int) >parm: MQ:Disable or enable Multiple Queues, default 1 (array >of >int) >parm: DCA:Disable or enable Direct Cache Access, 0=disabled, >1=descriptor only, 2=descriptor and data (array of int) >parm: RSS:Number of Receive-Side Scaling Descriptor Queues, >default 1=number of cpus (array of int) >parm: VMDQ:Number of Virtual Machine Device Queues: 0/1 = >disable, 2-16 enable (default=8) (array of int) >parm: max_vfs:Number of Virtual Functions: 0 = disable >(default), >1 = default settings, 2-63 = enable this many VFs (array of int) >parm: L2LBen:L2 Loopback Enable: 0 = disable, 1 = enable >(default) (array of int) >parm: InterruptThrottleRate:Maximum interrupts per second, per >vector, (0,1,956-488281), default 1 (array of int) >parm: LLIPort:Low Latency Interrupt TCP Port (0-65535) (array >of >int) >parm: LLIPush:Low Latency Interrupt on TCP Push flag (0,1) >(array of int) >parm: LLISize:Low Latency Interrupt on Packet Size (0-1500) >(array of int) >parm: LLIEType:Low Latency Interrupt Ethernet Protocol Type >(array of int) >parm: LLIVLANP:Low Latency Interrupt on VLAN priority >threshold >(array of int) >parm: RxBufferMode:0=1 descriptor per packet, > 1=use packet split, multiple descriptors per >jumbo frame > 2 (default)=use 1buf mode for 1500 mtu, packet >split for >jumbo (array of int) >parm: FdirMode:Flow Director filtering modes: > 0 = Filtering off > 1 = Signature Hashing filters (SW ATR) > 2 = Perfect Filters (array of int) >parm: FdirPballoc:Flow Director packet buffer allocation >level: > 1 = 8k hash filters or 2k perfect filters > 2 = 16k hash filters or 4k perfect filters > 3 = 32k hash filters or 8k perfect filters >(array of int) >parm: AtrSampleRate:Software ATR Tx packet sample rate (array >of int) >parm: FCoE:Disable or enable FCoE Offload, default 1 (array of >int) >parm: LRO:Large Receive Offload (0,1), default 1 = on (array >of int) > > > > > >------------------------------------------------------------------------ >------ >All the data continuously generated in your IT infrastructure >contains a definitive record of customers, application performance, >security threats, fraudulent activity, and more. Splunk takes this >data and makes sense of it. IT sense. And common sense. >http://p.sf.net/sfu/splunk-novd2d >_______________________________________________ >E1000-devel mailing list >[email protected] >https://lists.sourceforge.net/lists/listinfo/e1000-devel >To learn more about Intel® Ethernet, visit >http://communities.intel.com/community/wired ------------------------------------------------------------------------------ All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-novd2d _______________________________________________ E1000-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
