Are the packets being dropped by the HW or by the SW somewhere? Read the actual HW stats using ethtool to see if the HW is seeing all the packets.
Thanks. Cheers, John > -----Original Message----- > From: Kaushal Bhandankar [mailto:[email protected]] > Sent: Monday, June 17, 2013 7:32 PM > To: Skidmore, Donald C > Cc: [email protected] > Subject: Re: [E1000-devel] Intel 82599 Driver dropping packets Queue > Receive Descriptor Head > > Thanks for the reply. I ll elaborate the issue we are seeing. > When the affected queue started dropping packets, it had received 50% > of traffic comapred to any other queue which was working fine. After > this the traffic was diverted to another box. With no traffic in the > Box, the affected Queue drops 100 % packets it sees. We do not use > interrupt mechanism in our case. The Niantic hardware nic calculates > the hash and puts the packet in the appropriate queue. The user > Application directly read from this queue. There is no driver running > in Kernel space. This is done to improve performance of the system. > Also, We area creating a tap interface using ixgbe_tap_create call. > > ============= > > -bash-3.2# dmesg > q 15 > ide-gd driver 1.18 > Driver 'sd' needs updating - please use bus_type methods Driver 'sr' > needs updating - please use bus_type methods > ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver ehci_hcd > 0000:00:1a.7: PCI INT C -> GSI 18 (level, low) -> IRQ 18 ehci_hcd > 0000:00:1a.7: setting latency timer to 64 ehci_hcd 0000:00:1a.7: EHCI > Host Controller ehci_hcd 0000:00:1a.7: new USB bus registered, assigned > bus number 1 ehci_hcd 0000:00:1a.7: debug port 1 ehci_hcd 0000:00:1a.7: > cache line size of 32 is not supported ehci_hcd 0000:00:1a.7: irq 18, > io mem 0xdbbde000 ehci_hcd 0000:00:1a.7: USB 2.0 started, EHCI 1.00 usb > usb1: configuration #1 chosen from 1 choice hub 1-0:1.0: USB hub found > hub 1-0:1.0: 6 ports detected ehci_hcd 0000:00:1d.7: PCI INT A -> GSI > 16 (level, low) -> IRQ 16 ehci_hcd 0000:00:1d.7: setting latency timer > to 64 ehci_hcd 0000:00:1d.7: EHCI Host Controller ehci_hcd > 0000:00:1d.7: new USB bus registered, assigned bus number 2 ehci_hcd > 0000:00:1d.7: debug port 1 ehci_hcd 0000:00:1d.7: cache line size of 32 > is not supported ehci_hcd 0000:00:1d.7: irq 16, io mem 0xdbbdf000 > ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00 usb usb2: > configuration #1 chosen from 1 choice hub 2-0:1.0: USB hub found hub 2- > 0:1.0: 6 ports detected > ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver > uhci_hcd: USB Universal Host Controller Interface driver uhci_hcd > 0000:00:1a.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16 uhci_hcd > 0000:00:1a.0: setting latency timer to 64 uhci_hcd 0000:00:1a.0: UHCI > Host Controller uhci_hcd 0000:00:1a.0: new USB bus registered, assigned > bus number 3 uhci_hcd 0000:00:1a.0: irq 16, io base 0x0000ac20 usb > usb3: configuration #1 chosen from 1 choice hub 3-0:1.0: USB hub found > hub 3-0:1.0: 2 ports detected uhci_hcd 0000:00:1a.1: PCI INT B -> GSI > 17 (level, low) -> IRQ 17 uhci_hcd 0000:00:1a.1: setting latency timer > to 64 uhci_hcd 0000:00:1a.1: UHCI Host Controller uhci_hcd > 0000:00:1a.1: new USB bus registered, assigned bus number 4 uhci_hcd > 0000:00:1a.1: irq 17, io base 0x0000ac40 usb usb4: configuration #1 > chosen from 1 choice hub 4-0:1.0: USB hub found hub 4-0:1.0: 2 ports > detected uhci_hcd 0000:00:1a.2: PCI INT C -> GSI 18 (level, low) -> IRQ > 18 uhci_hcd 0000:00:1a.2: setting latency timer to 64 uhci_hcd > 0000:00:1a.2: UHCI Host Controller uhci_hcd 0000:00:1a.2: new USB bus > registered, assigned bus number 5 uhci_hcd 0000:00:1a.2: irq 18, io > base 0x0000ac60 usb usb5: configuration #1 chosen from 1 choice hub 5- > 0:1.0: USB hub found hub 5-0:1.0: 2 ports detected uhci_hcd > 0000:00:1d.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16 uhci_hcd > 0000:00:1d.0: setting latency timer to 64 uhci_hcd 0000:00:1d.0: UHCI > Host Controller uhci_hcd 0000:00:1d.0: new USB bus registered, assigned > bus number 6 uhci_hcd 0000:00:1d.0: irq 16, io base 0x0000ac80 usb > usb6: configuration #1 chosen from 1 choice hub 6-0:1.0: USB hub found > hub 6-0:1.0: 2 ports detected uhci_hcd 0000:00:1d.1: PCI INT B -> GSI > 17 (level, low) -> IRQ 17 uhci_hcd 0000:00:1d.1: setting latency timer > to 64 uhci_hcd 0000:00:1d.1: UHCI Host Controller uhci_hcd > 0000:00:1d.1: new USB bus registered, assigned bus number 7 uhci_hcd > 0000:00:1d.1: irq 17, io base 0x0000aca0 usb usb7: configuration #1 > chosen from 1 choice hub 7-0:1.0: USB hub found hub 7-0:1.0: 2 ports > detected uhci_hcd 0000:00:1d.2: PCI INT C -> GSI 18 (level, low) -> IRQ > 18 uhci_hcd 0000:00:1d.2: setting latency timer to 64 uhci_hcd > 0000:00:1d.2: UHCI Host Controller uhci_hcd 0000:00:1d.2: new USB bus > registered, assigned bus number 8 uhci_hcd 0000:00:1d.2: irq 18, io > base 0x0000acc0 usb usb8: configuration #1 chosen from 1 choice hub 8- > 0:1.0: USB hub found hub 8-0:1.0: 2 ports detected Initializing USB > Mass Storage driver... > usbcore: registered new interface driver usb-storage USB Mass Storage > support registered. > PNP: No PS/2 controller found. Probing ports directly. > i8042.c: No controller found. > mice: PS/2 mouse device common for all mice rtc_cmos 00:09: RTC can > wake from S4 rtc_cmos 00:09: rtc core: registered rtc_cmos as rtc0 > rtc0: alarms up to one day, 114 bytes nvram i2c /dev entries driver > i801_smbus 0000:00:1f.3: PCI INT C -> GSI 18 (level, low) -> IRQ 18 > device-mapper: ioctl: 4.14.0-ioctl (2008-04-23) initialised: > [email protected] > cpuidle: using governor ladder > ioatdma 0000:00:16.0: can't derive routing for PCI INT A ioatdma > 0000:00:16.0: PCI INT A: no GSI ioatdma 0000:00:16.0: setting latency > timer to 64 ioatdma 0000:00:16.0: Intel(R) I/OAT DMA Engine found, 1 > channels, device version 0x30, driver version 3.64 ioatdma > 0000:00:16.0: irq 68 for MSI/MSI-X ioatdma 0000:00:16.0: DCA is > disabled in BIOS ioatdma 0000:00:16.1: can't derive routing for PCI INT > B ioatdma 0000:00:16.1: PCI INT B: no GSI ioatdma 0000:00:16.1: setting > latency timer to 64 ioatdma 0000:00:16.1: Intel(R) I/OAT DMA Engine > found, 1 channels, device version 0x30, driver version 3.64 ioatdma > 0000:00:16.1: irq 69 for MSI/MSI-X ioatdma 0000:00:16.1: DCA is > disabled in BIOS ioatdma 0000:00:16.2: can't derive routing for PCI INT > C ioatdma 0000:00:16.2: PCI INT C: no GSI ioatdma 0000:00:16.2: setting > latency timer to 64 ioatdma 0000:00:16.2: Intel(R) I/OAT DMA Engine > found, 1 channels, device version 0x30, driver version 3.64 ioatdma > 0000:00:16.2: irq 70 for MSI/MSI-X Clocksource tsc unstable (delta = > 4687261807 ns) usb 1-4: new high speed USB device using ehci_hcd and > address 2 ioatdma 0000:00:16.2: DCA is disabled in BIOS ioatdma > 0000:00:16.3: can't derive routing for PCI INT D ioatdma 0000:00:16.3: > PCI INT D: no GSI ioatdma 0000:00:16.3: setting latency timer to 64 > ioatdma 0000:00:16.3: Intel(R) I/OAT DMA Engine found, 1 channels, > device version 0x30, driver version 3.64 ioatdma 0000:00:16.3: irq 71 > for MSI/MSI-X ioatdma 0000:00:16.3: DCA is disabled in BIOS ioatdma > 0000:00:16.4: can't derive routing for PCI INT A ioatdma 0000:00:16.4: > PCI INT A: no GSI ioatdma 0000:00:16.4: setting latency timer to 64 > ioatdma 0000:00:16.4: Intel(R) I/OAT DMA Engine found, 1 channels, > device version 0x30, driver version 3.64 ioatdma 0000:00:16.4: irq 72 > for MSI/MSI-X usb 1-4: configuration #1 chosen from 1 choice > scsi0 : SCSI emulation for USB Mass Storage devices ioatdma > 0000:00:16.4: DCA is disabled in BIOS ioatdma 0000:00:16.5: can't > derive routing for PCI INT B ioatdma 0000:00:16.5: PCI INT B: no GSI > ioatdma 0000:00:16.5: setting latency timer to 64 ioatdma 0000:00:16.5: > Intel(R) I/OAT DMA Engine found, 1 channels, device version 0x30, > driver version 3.64 ioatdma 0000:00:16.5: irq 73 for MSI/MSI-X ioatdma > 0000:00:16.5: DCA is disabled in BIOS ioatdma 0000:00:16.6: can't > derive routing for PCI INT C ioatdma 0000:00:16.6: PCI INT C: no GSI > ioatdma 0000:00:16.6: setting latency timer to 64 ioatdma 0000:00:16.6: > Intel(R) I/OAT DMA Engine found, 1 channels, device version 0x30, > driver version 3.64 ioatdma 0000:00:16.6: irq 74 for MSI/MSI-X ioatdma > 0000:00:16.6: DCA is disabled in BIOS ioatdma 0000:00:16.7: can't > derive routing for PCI INT D ioatdma 0000:00:16.7: PCI INT D: no GSI > ioatdma 0000:00:16.7: setting latency timer to 64 ioatdma 0000:00:16.7: > Intel(R) I/OAT DMA Engine found, 1 channels, device version 0x30, > driver version 3.64 ioatdma 0000:00:16.7: irq 75 for MSI/MSI-X ioatdma > 0000:00:16.7: DCA is disabled in BIOS > usbcore: registered new interface driver usbhid > usbhid: v2.6:USB HID core driver > TCP cubic registered > IPv6: Loaded, but is disabled by default. IPv6 may be enabled on > individual interfaces. > NET: Registered protocol family 10 > NET: Registered protocol family 17 > NET: Registered protocol family 5 > usb-storage: device found at 2 > usb-storage: waiting for device to settle before scanning rtc_cmos > 00:09: setting system clock to 2013-04-24 08:30:08 UTC (1366792208) > Freeing unused kernel memory: 1032k freed Write protecting the kernel > read-only data: 4272k > scsi 0:0:0:0: Direct-Access Unigen CEU2000 1100 PQ: 0 > ANSI: > 0 CCS > sd 0:0:0:0: [sda] 4014080 512-byte hardware sectors: (2.05 GB/1.91 GiB) > sd 0:0:0:0: [sda] Write Protect is off sd 0:0:0:0: [sda] Mode Sense: 43 > 00 00 00 sd 0:0:0:0: [sda] Assuming drive cache: write through sd > 0:0:0:0: [sda] 4014080 512-byte hardware sectors: (2.05 GB/1.91 GiB) sd > 0:0:0:0: [sda] Write Protect is off sd 0:0:0:0: [sda] Mode Sense: 43 00 > 00 00 sd 0:0:0:0: [sda] Assuming drive cache: write through > sda: sda1 sda2 sda3 sda4 > sd 0:0:0:0: [sda] Attached SCSI removable disk sd 0:0:0:0: Attached > scsi generic sg0 type 0 > usb-storage: device scan complete > kjournald starting. Commit interval 5 seconds > EXT3-fs: mounted filesystem with ordered data mode. > kjournald starting. Commit interval 5 seconds > EXT3 FS on sda2, internal journal > EXT3-fs: mounted filesystem with ordered data mode. > kjournald starting. Commit interval 5 seconds > EXT3-fs: mounted filesystem with ordered data mode. > linux_kernel_bde: module license 'Proprietary' taints kernel. > linux-kernel-bde 0000:12:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ > 16 linux-kernel-bde (25599): ** Alloc contiguous DMA-able memroy > 8388608 bytes =======> totalram(6148886) PAGE_SHIFT(12) > blk_size(524288) order(7) > PAGE_SIZE(4096) > ==========> 24 > linux-kernel-bde (25599): alloc_mpool : dma_pbase (0xd7400000) > dma_vbase > (ffff8803d7400000) > EXT3 FS on sda1, internal journal > cpp INFO at cppInitBusDevice:862: Detected Tarari device 0, deviceId > 0x0a05 on bus=11, slot=0 pci 0000:0b:00.0: PCI INT A -> GSI 16 (level, > low) -> IRQ 16 pci 0000:0b:00.0: irq 76 for MSI/MSI-X > IHM: Initializing Interface Helper Module > IHM: registering chr device > Module registered 248, from (pid 26094) > Probe called with class 0x0 > Bus 8 Dev 0 Func 0 > ihm 0000:08:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16 ihm > 0000:08:00.0: setting latency timer to 64 Probe called with class 0x0 > Bus 8 Dev 0 Func 1 ihm 0000:08:00.1: PCI INT B -> GSI 17 (level, low) - > > IRQ 17 ihm 0000:08:00.1: setting latency timer to 64 Probe called > with class 0x0 Bus 10 Dev 0 Func 0 ihm 0000:0a:00.0: PCI INT A -> GSI > 16 (level, low) -> IRQ 16 ihm 0000:0a:00.0: setting latency timer to 64 > Probe called with class 0x0 Bus 10 Dev 0 Func 1 ihm 0000:0a:00.1: PCI > INT B -> GSI 17 (level, low) -> IRQ 17 ihm 0000:0a:00.1: setting > latency timer to 64 > ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version > 2.0.38.3-NAPI Copyright (c) 1999-2009 Intel Corporation. > IHM: Remove called > ihm 0000:08:00.1: PCI INT B disabled > ixgbe 0000:08:00.1: PCI INT B -> GSI 17 (level, low) -> IRQ 17 ixgbe > 0000:08:00.1: enabling bus mastering ixgbe 0000:08:00.1: setting > latency timer to 64 > ixgbe: 0000:08:00.1: ixgbe_check_options: Flow Director hash filtering > enabled > ixgbe: 0000:08:00.1: ixgbe_check_options: Flow Director allocated 64kB > of packet buffer > ixgbe: 0000:08:00.1: ixgbe_check_options: ATR Tx Packet sample rate set > to default of 20 ixgbe 0000:08:00.1: irq 77 for MSI/MSI-X ixgbe > 0000:08:00.1: irq 78 for MSI/MSI-X ixgbe 0000:08:00.1: irq 79 for > MSI/MSI-X ixgbe 0000:08:00.1: irq 80 for MSI/MSI-X ixgbe 0000:08:00.1: > irq 81 for MSI/MSI-X ixgbe 0000:08:00.1: irq 82 for MSI/MSI-X ixgbe > 0000:08:00.1: irq 83 for MSI/MSI-X ixgbe 0000:08:00.1: irq 84 for > MSI/MSI-X ixgbe 0000:08:00.1: irq 85 for MSI/MSI-X ixgbe 0000:08:00.1: > irq 86 for MSI/MSI-X ixgbe 0000:08:00.1: irq 87 for MSI/MSI-X ixgbe > 0000:08:00.1: irq 88 for MSI/MSI-X ixgbe 0000:08:00.1: irq 89 for > MSI/MSI-X ixgbe 0000:08:00.1: irq 90 for MSI/MSI-X ixgbe 0000:08:00.1: > irq 91 for MSI/MSI-X ixgbe 0000:08:00.1: irq 92 for MSI/MSI-X ixgbe > 0000:08:00.1: irq 93 for MSI/MSI-X ixgbe 0000:08:00.1: irq 94 for > MSI/MSI-X ixgbe 0000:08:00.1: irq 95 for MSI/MSI-X ixgbe 0000:08:00.1: > irq 96 for MSI/MSI-X ixgbe 0000:08:00.1: irq 97 for MSI/MSI-X ixgbe > 0000:08:00.1: irq 98 for MSI/MSI-X ixgbe 0000:08:00.1: irq 99 for > MSI/MSI-X ixgbe 0000:08:00.1: irq 100 for MSI/MSI-X ixgbe 0000:08:00.1: > irq 101 for MSI/MSI-X ixgbe 0000:08:00.1: irq 102 for MSI/MSI-X ixgbe > 0000:08:00.1: irq 103 for MSI/MSI-X ixgbe 0000:08:00.1: irq 104 for > MSI/MSI-X ixgbe 0000:08:00.1: irq 105 for MSI/MSI-X ixgbe 0000:08:00.1: > irq 106 for MSI/MSI-X ixgbe 0000:08:00.1: irq 107 for MSI/MSI-X ixgbe > 0000:08:00.1: irq 108 for MSI/MSI-X ixgbe 0000:08:00.1: irq 109 for > MSI/MSI-X > ixgbe: 0000:08:00.1: ixgbe_init_interrupt_scheme: Multiqueue Enabled: > Rx Queue count = 16, Tx Queue count = 16 > ixgbe: 0000:08:00.1: ixgbe_probe: Double Vlan Mode not Enabled > ixgbe: eth0: ixgbe_probe: No DCA provider found. Please start ioatdma > for DCA functionality. > ixgbe: eth0: ixgbe_probe: (PCI Express:5.0Gb/s:Width x8) > 00:00:00:01:00:02 > ixgbe: eth0: ixgbe_probe: MAC: 2, PHY: 1, PBA No: ffffff-0ff > ixgbe: eth0: ixgbe_probe: LRO is disabled > ixgbe: eth0: ixgbe_probe: HW RSC is enabled > ixgbe: eth0: ixgbe_probe: Intel(R) 10 Gigabit Network Connection > IHM: Remove called > ihm 0000:0a:00.0: PCI INT A disabled > e1000e: Intel(R) PRO/1000 Network Driver - 0.3.3.3-k6 > e1000e: Copyright (c) 1999-2008 Intel Corporation. > e1000e 0000:10:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16 e1000e > 0000:10:00.0: setting latency timer to 64 e1000e 0000:10:00.0: irq 110 > for MSI/MSI-X e1000e 0000:10:00.0: irq 111 for MSI/MSI-X e1000e > 0000:10:00.0: irq 112 for MSI/MSI-X > 0000:10:00.0: eth1: (PCI Express:2.5GB/s:Width x1) 70:ca:9b:f0:4f:85 > 0000:10:00.0: eth1: Intel(R) PRO/1000 Network Connection > 0000:10:00.0: eth1: MAC: 3, PHY: 8, PBA No: 1020ff-0ff e1000e > 0000:11:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16 e1000e > 0000:11:00.0: setting latency timer to 64 e1000e 0000:11:00.0: irq 113 > for MSI/MSI-X e1000e 0000:11:00.0: irq 114 for MSI/MSI-X e1000e > 0000:11:00.0: irq 115 for MSI/MSI-X > 0000:11:00.0: eth2: (PCI Express:2.5GB/s:Width x1) 70:ca:9b:f0:4f:84 > 0000:11:00.0: eth2: Intel(R) PRO/1000 Network Connection > 0000:11:00.0: eth2: MAC: 3, PHY: 8, PBA No: 1020ff-0ff > Intel(R) Gigabit Ethernet Network Driver - version 1.2.45-k2 Copyright > (c) 2008 Intel Corporation. > ip_tables: (C) 2000-2006 Netfilter Core Team nf_conntrack version 0.5.0 > (16384 buckets, 65536 max) CONFIG_NF_CT_ACCT is deprecated and will be > removed soon. Please use > nf_conntrack.acct=1 kernel paramater, acct=1 nf_conntrack module option > or sysctl net.netfilter.nf_conntrack_acct=1 to enable it. > ixgbe: sy0_0: ixgbe_watchdog_task: NIC Link is Up 10 Gbps, Flow > Control: > RX/TX > ADDRCONF(NETDEV_UP): sy0_0: link is not ready > ADDRCONF(NETDEV_CHANGE): sy0_0: link becomes ready > TIPC: Activated (version 1.6.4 compiled Apr 24 2012 03:29:39) > NET: Registered protocol family 30 > TIPC: Started in single node mode > TIPC: Started in network mode > TIPC: Own node address <1.1.2>, network identity 1234 > TIPC: Enabled bearer <eth:sy0_0>, discovery domain <1.1.0>, priority 10 > TIPC: Established link <1.1.2:sy0_0-1.1.1:tap0> on network plane A > ADDRCONF(NETDEV_UP): ma0_0: link is not ready > e1000e: ma0_0 NIC Link is Up 100 Mbps Full Duplex, Flow Control: None > 0000:11:00.0: ma0_0: 10/100 speed: disabling TSO > ADDRCONF(NETDEV_CHANGE): ma0_0: link becomes ready device po0_0 entered > promiscuous mode device po0_0 left promiscuous mode device po0_0 > entered promiscuous mode device po0_0 left promiscuous mode device > po0_0 entered promiscuous mode device po0_0 left promiscuous mode > device po0_0 entered promiscuous mode device po0_0 left promiscuous > mode device po0_0 entered promiscuous mode device po0_0 left > promiscuous mode device po0_0 entered promiscuous mode device po0_0 > left promiscuous mode device po0_0 entered promiscuous mode device > po0_0 left promiscuous mode device po0_0 entered promiscuous mode > device po0_0 left promiscuous mode device po0_0 entered promiscuous > mode device po0_0 left promiscuous mode -bash-3.2# su - cisco > CLMBOH0001-IPS903# show int port > portChannel portChannel0/0 > CLMBOH0001-IPS903# show int portChannel > MAC statistics from interface PortChannel0/0 > Interface function = Sensing interface > Description = > Media Type = backplane > Default Vlan = 0 > InlineMode = Unpaired > Pair Status = N/A > Hardware Bypass Capable = No > Hardware Bypass Paired = N/A > Link Status = Up > Admin Enabled Status = Enabled > Link Speed = Auto_10000 > Link Duplex = Auto_Full > Missed Packet Percentage = 0 > Total Packets Received = 3479352341 > Total Bytes Received = 3963816352204 > Total Multicast Packets Received = 0 > Total Broadcast Packets Received = 0 > Total Jumbo Packets Received = 877491302 > Total Undersize Packets Received = 0 > Total Receive Errors = 2498336 > Total Receive FIFO Overruns = 0 > Total requests for buffer when none available = 0 > Total Packets Transmitted = 3479352325 > Total Bytes Transmitted = 3963816303436 > Total Multicast Packets Transmitted = 0 > Total Broadcast Packets Transmitted = 0 > Total Jumbo Packets Transmitted = 877491301 > Total Undersize Packets Transmitted = 0 > Total Transmit Errors = 0 > Total Transmit FIFO Overruns = 0 > CLMBOH0001-IPS903# exit > > ================== > > exec: /sbin/lspci -s 08:0.0 -vvv > 08:00.0 Class 0200: Device 8086:10fc (rev 01) > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- DisINTx- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- > <MAbort- >SERR- <PERR- INTx- > Latency: 0, Cache Line Size: 64 bytes > Interrupt: pin A routed to IRQ 16 > Region 0: Memory at decc0000 (64-bit, non-prefetchable) [size=128K] > Region 2: I/O ports at fcc0 [size=32] Region 4: Memory at decb8000 (64- > bit, non-prefetchable) [size=16K] > Capabilities: [40] Power Management version 3 > Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2- > ,D3hot+,D3cold-) > Status: D0 PME-Enable- DSel=0 DScale=1 PME- > Capabilities: [50] Message Signalled Interrupts: Mask+ 64bit+ Count=1/1 > Enable- > Address: 0000000000000000 Data: 0000 > Masking: 00000000 Pending: 00000000 > Capabilities: [70] MSI-X: Enable- Mask- TabSize=64 Vector table: BAR=4 > offset=00000000 > PBA: BAR=4 offset=00002000 > Capabilities: [a0] Express (v2) Endpoint, MSI 00 DevCap:MaxPayload 512 > bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ DevCtl:Report errors: > Correctable- Non-Fatal- Fatal- Unsupported- > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset- > MaxPayload 256 bytes, MaxReadReq 512 bytes DevSta:CorrErr+ UncorrErr- > FatalErr- UnsuppReq+ AuxPwr- TransPend- LnkCap:Port #0, Speed 5GT/s, > Width x8, ASPM L0s, Latency L0 unlimited, L1 <32us > ClockPM- Suprise- LLActRep- BwNot- > LnkCtl:ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta:Speed 5GT/s, > Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- > ABWMgmt- > DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- > LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, > Selectable > De-emphasis: -6dB > Transmit Margin: Normal Operating Range, EnterModifiedCompliance- > ComplianceSOS- > Compliance De-emphasis: -6dB > LnkSta2: Current De-emphasis Level: -6dB > Capabilities: [e0] Vital Product Data <?> > Capabilities: [100] Advanced Error Reporting > UESta:DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- > ECRC- UnsupReq- ACSVoil- > UEMsk:DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- > ECRC- UnsupReq- ACSVoil- > UESvrt:DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ > MalfTLP+ > ECRC- UnsupReq- ACSVoil- > CESta:RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ > CESta:RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ > AERCap:First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn- > Capabilities: [140] Device Serial Number 01-00-01-ff-ff-00-00-00 > Capabilities: [150] Alternative Routing-ID Interpretation (ARI) > ARICap:MFVC- ACS-, Next Function: 1 > ARICtl:MFVC- ACS-, Function Group: 0 > Capabilities: [160] Single Root I/O Virtualization (SR-IOV) > IOVCap:Migration-, Interrupt Message Number: 000 > IOVCtl:Enable- Migration- Interrupt- MSE- ARIHierarchy- > IOVSta:Migration- > Initial VFs: 64, Total VFs: 64, Number of VFs: 0, Function Dependency > Link: > 00 > VF offset: 384, stride: 2, Device ID: 10ed Supported Page Size: > 00000553, System Page Size: 00000001 VF Migration: offset: 00000000, > BIR: 1 Kernel driver in use: ihm Kernel modules: ixgbe > exec: /sbin/lspci -s 08:0.0 -vvv > 08:00.0 Class 0200: Device 8086:10fc (rev 01) > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- DisINTx- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- > <MAbort- >SERR- <PERR- INTx- > Latency: 0, Cache Line Size: 64 bytes > Interrupt: pin A routed to IRQ 16 > Region 0: Memory at decc0000 (64-bit, non-prefetchable) [size=128K] > Region 2: I/O ports at fcc0 [size=32] Region 4: Memory at decb8000 (64- > bit, non-prefetchable) [size=16K] > Capabilities: [40] Power Management version 3 > Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2- > ,D3hot+,D3cold-) > Status: D0 PME-Enable- DSel=0 DScale=1 PME- > Capabilities: [50] Message Signalled Interrupts: Mask+ 64bit+ Count=1/1 > Enable- > Address: 0000000000000000 Data: 0000 > Masking: 00000000 Pending: 00000000 > Capabilities: [70] MSI-X: Enable- Mask- TabSize=64 Vector table: BAR=4 > offset=00000000 > PBA: BAR=4 offset=00002000 > Capabilities: [a0] Express (v2) Endpoint, MSI 00 DevCap:MaxPayload 512 > bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ DevCtl:Report errors: > Correctable- Non-Fatal- Fatal- Unsupported- > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset- > MaxPayload 256 bytes, MaxReadReq 512 bytes DevSta:CorrErr+ UncorrErr- > FatalErr- UnsuppReq+ AuxPwr- TransPend- LnkCap:Port #0, Speed 5GT/s, > Width x8, ASPM L0s, Latency L0 unlimited, L1 <32us > ClockPM- Suprise- LLActRep- BwNot- > LnkCtl:ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta:Speed 5GT/s, > Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- > ABWMgmt- > DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- > LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, > Selectable > De-emphasis: -6dB > Transmit Margin: Normal Operating Range, EnterModifiedCompliance- > ComplianceSOS- > Compliance De-emphasis: -6dB > LnkSta2: Current De-emphasis Level: -6dB > Capabilities: [e0] Vital Product Data <?> > Capabilities: [100] Advanced Error Reporting > UESta:DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- > ECRC- UnsupReq- ACSVoil- > UEMsk:DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- > ECRC- UnsupReq- ACSVoil- > UESvrt:DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ > MalfTLP+ > ECRC- UnsupReq- ACSVoil- > CESta:RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ > CESta:RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ > AERCap:First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn- > Capabilities: [140] Device Serial Number 01-00-01-ff-ff-00-00-00 > Capabilities: [150] Alternative Routing-ID Interpretation (ARI) > ARICap:MFVC- ACS-, Next Function: 1 > ARICtl:MFVC- ACS-, Function Group: 0 > Capabilities: [160] Single Root I/O Virtualization (SR-IOV) > IOVCap:Migration-, Interrupt Message Number: 000 > IOVCtl:Enable- Migration- Interrupt- MSE- ARIHierarchy- > IOVSta:Migration- > Initial VFs: 64, Total VFs: 64, Number of VFs: 0, Function Dependency > Link: > 00 > VF offset: 384, stride: 2, Device ID: 10ed Supported Page Size: > 00000553, System Page Size: 00000001 VF Migration: offset: 00000000, > BIR: 1 Kernel driver in use: ihm Kernel modules: ixgbe > exec: /sbin/lspci -s 0a:0.0 -vvv > 0a:00.0 Class 0200: Device 8086:10fc (rev 01) > Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- DisINTx- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- > <MAbort- >SERR- <PERR- INTx- > Interrupt: pin A routed to IRQ 16 > Region 0: Memory at de5c0000 (64-bit, non-prefetchable) [size=128K] > Region 2: I/O ports at ecc0 [size=32] Region 4: Memory at de5b8000 (64- > bit, non-prefetchable) [size=16K] > Capabilities: [40] Power Management version 3 > Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2- > ,D3hot+,D3cold-) > Status: D0 PME-Enable- DSel=0 DScale=1 PME- > Capabilities: [50] Message Signalled Interrupts: Mask+ 64bit+ Count=1/1 > Enable- > Address: 0000000000000000 Data: 0000 > Masking: 00000000 Pending: 00000000 > Capabilities: [70] MSI-X: Enable- Mask- TabSize=64 Vector table: BAR=4 > offset=00000000 > PBA: BAR=4 offset=00002000 > Capabilities: [a0] Express (v2) Endpoint, MSI 00 DevCap:MaxPayload 512 > bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ DevCtl:Report errors: > Correctable- Non-Fatal- Fatal- Unsupported- > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset- > MaxPayload 256 bytes, MaxReadReq 512 bytes DevSta:CorrErr+ UncorrErr- > FatalErr- UnsuppReq+ AuxPwr- TransPend- LnkCap:Port #0, Speed 5GT/s, > Width x8, ASPM L0s, Latency L0 unlimited, L1 <32us > ClockPM- Suprise- LLActRep- BwNot- > LnkCtl:ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta:Speed 5GT/s, > Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- > ABWMgmt- > DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- > LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, > Selectable > De-emphasis: -6dB > Transmit Margin: Normal Operating Range, EnterModifiedCompliance- > ComplianceSOS- > Compliance De-emphasis: -6dB > LnkSta2: Current De-emphasis Level: -6dB > Capabilities: [e0] Vital Product Data <?> > Capabilities: [100] Advanced Error Reporting > UESta:DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- > ECRC- UnsupReq- ACSVoil- > UEMsk:DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- > ECRC- UnsupReq- ACSVoil- > UESvrt:DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ > MalfTLP+ > ECRC- UnsupReq- ACSVoil- > CESta:RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ > CESta:RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ > AERCap:First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn- > Capabilities: [140] Device Serial Number 03-00-01-ff-ff-00-00-00 > Capabilities: [150] Alternative Routing-ID Interpretation (ARI) > ARICap:MFVC- ACS-, Next Function: 1 > ARICtl:MFVC- ACS-, Function Group: 0 > Capabilities: [160] Single Root I/O Virtualization (SR-IOV) > IOVCap:Migration-, Interrupt Message Number: 000 > IOVCtl:Enable- Migration- Interrupt- MSE- ARIHierarchy- > IOVSta:Migration- > Initial VFs: 64, Total VFs: 64, Number of VFs: 0, Function Dependency > Link: > 00 > VF offset: 384, stride: 2, Device ID: 10ed Supported Page Size: > 00000553, System Page Size: 00000001 VF Migration: offset: 00000000, > BIR: 1 Kernel modules: ixgbe > exec: /sbin/lspci -s 09:0.0 -vvv > exec: /sbin/lspci -s 0b:0.0 -vvv > 0b:00.0 Class 0b40: Device 1000:0a05 (rev 01) > Subsystem: Device 1000:0a0a > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- > <MAbort- >SERR- <PERR- INTx- > Latency: 0, Cache Line Size: 64 bytes > Interrupt: pin A routed to IRQ 76 > Region 0: Memory at de200000 (64-bit, non-prefetchable) [size=1M] > Capabilities: [40] Power Management version 3 > Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA PME(D0+,D1+,D2- > ,D3hot+,D3cold-) > Status: D0 PME-Enable- DSel=0 DScale=0 PME- > Capabilities: [50] Message Signalled Interrupts: Mask- 64bit+ Count=1/1 > Enable+ > Address: 00000000fee00000 Data: 409a > Capabilities: [70] Express (v2) Endpoint, MSI 00 DevCap:MaxPayload 256 > bytes, PhantFunc 0, Latency L0s <128ns, L1 <2us > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- DevCtl:Report errors: > Correctable- Non-Fatal- Fatal- Unsupported- > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- > MaxPayload 128 bytes, MaxReadReq 256 bytes > DevSta:CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend+ > LnkCap:Port #0, Speed 2.5GT/s, Width x8, ASPM L0s L1, Latency L0 > <512ns, L1 <64us > ClockPM- Suprise- LLActRep- BwNot- > LnkCtl:ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta:Speed 2.5GT/s, > Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- > ABWMgmt- > DevCap2: Completion Timeout: Not Supported, TimeoutDis+ > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- > LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, > Selectable > De-emphasis: -6dB > Transmit Margin: Normal Operating Range, EnterModifiedCompliance- > ComplianceSOS- > Compliance De-emphasis: -6dB > LnkSta2: Current De-emphasis Level: -6dB > Capabilities: [b0] MSI-X: Enable- Mask- TabSize=33 Vector table: BAR=0 > offset=00000000 > PBA: BAR=0 offset=00000000 > Capabilities: [100] Advanced Error Reporting > UESta:DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- > ECRC- UnsupReq- ACSVoil- > UEMsk:DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- > ECRC- UnsupReq- ACSVoil- > UESvrt:DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ > MalfTLP+ > ECRC- UnsupReq- ACSVoil- > CESta:RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ > CESta:RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ > AERCap:First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- > Capabilities: [140] Virtual Channel <?> > exec: /sbin/lspci -s 0c:0.0 -vvv > 0c:00.0 Class 0100: Device 1000:0056 (rev 10) > Subsystem: Device 1000:3180 > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- DisINTx- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- > <MAbort- >SERR- <PERR- INTx- > Latency: 0, Cache Line Size: 64 bytes > Interrupt: pin A routed to IRQ 5 > Region 0: I/O ports at dc00 [size=256] > Region 1: Memory at dddec000 (64-bit, non-prefetchable) [size=16K] > Region 3: Memory at dddf0000 (64-bit, non-prefetchable) [size=64K] > Expansion ROM at dde00000 [disabled] [size=2M] > Capabilities: [50] Power Management version 2 > Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot- > ,D3cold-) > Status: D0 PME-Enable- DSel=0 DScale=0 PME- > Capabilities: [68] Express (v1) Endpoint, MSI 00 DevCap:MaxPayload 4096 > bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us > ExtTag+ AttnBtn- AttnInd- PwrInd- RBE- FLReset- > DevCtl:Report errors: Correctable- Non-Fatal- Fatal- Unsupported- > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ > MaxPayload 128 bytes, MaxReadReq 512 bytes > DevSta:CorrErr- UncorrErr+ FatalErr- UnsuppReq+ AuxPwr- TransPend- > LnkCap:Port #0, Speed 2.5GT/s, Width x8, ASPM L0s L1, Latency L0 <64ns, > L1 <1us > ClockPM- Suprise- LLActRep- BwNot- > LnkCtl:ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta:Speed 2.5GT/s, > Width x4, TrErr- Train- SlotClk- DLActive- BWMgmt- > ABWMgmt- > Capabilities: [98] Message Signalled Interrupts: Mask- 64bit+ Count=1/1 > Enable- > Address: 0000000000000000 Data: 0000 > Capabilities: [b0] MSI-X: Enable- Mask- TabSize=1 Vector table: BAR=1 > offset=00002000 > PBA: BAR=1 offset=00003000 > Capabilities: [100] Advanced Error Reporting > UESta:DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- > ECRC- UnsupReq+ ACSVoil- > UEMsk:DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- > ECRC- UnsupReq- ACSVoil- > UESvrt:DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ > MalfTLP+ > ECRC- UnsupReq- ACSVoil- > CESta:RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- > CESta:RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- > AERCap:First Error Pointer: 14, GenCap+ CGenEn- ChkCap+ ChkEn- > exec: /sbin/lspci -s 10:0.0 -vvv > 10:00.0 Class 0200: Device 8086:10d3 > Subsystem: Device 8086:0000 > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- > <MAbort- >SERR- <PERR- INTx- > Latency: 0, Cache Line Size: 64 bytes > Interrupt: pin A routed to IRQ 16 > Region 0: Memory at dcee0000 (32-bit, non-prefetchable) [size=128K] > Region 2: I/O ports at cce0 [size=32] Region 3: Memory at dcedc000 (32- > bit, non-prefetchable) [size=16K] > Capabilities: [c8] Power Management version 2 > Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2- > ,D3hot+,D3cold+) > Status: D0 PME-Enable- DSel=0 DScale=1 PME- > Capabilities: [d0] Message Signalled Interrupts: Mask- 64bit+ Count=1/1 > Enable- > Address: 0000000000000000 Data: 0000 > Capabilities: [e0] Express (v1) Endpoint, MSI 00 DevCap:MaxPayload 256 > bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- DevCtl:Report errors: > Correctable- Non-Fatal- Fatal- Unsupported- > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ > MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta:CorrErr+ UncorrErr- > FatalErr- UnsuppReq+ AuxPwr+ TransPend- LnkCap:Port #0, Speed 2.5GT/s, > Width x1, ASPM L0s L1, Latency L0 <128ns, L1 <64us > ClockPM- Suprise- LLActRep- BwNot- > LnkCtl:ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta:Speed 2.5GT/s, > Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- > ABWMgmt- > Capabilities: [a0] MSI-X: Enable+ Mask- TabSize=5 Vector table: BAR=3 > offset=00000000 > PBA: BAR=3 offset=00002000 > Capabilities: [100] Advanced Error Reporting > UESta:DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- > ECRC- UnsupReq- ACSVoil- > UEMsk:DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- > ECRC- UnsupReq- ACSVoil- > UESvrt:DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ > MalfTLP+ > ECRC- UnsupReq- ACSVoil- > CESta:RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ > CESta:RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ > AERCap:First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- > Capabilities: [140] Device Serial Number 85-4f-f0-ff-ff-9b-ca-70 Kernel > driver in use: e1000e Kernel modules: e1000e, e1000-cids > exec: /sbin/lspci -s 11:0.0 -vvv > 11:00.0 Class 0200: Device 8086:10d3 > Subsystem: Device 8086:0000 > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- > <MAbort- >SERR- <PERR- INTx- > Latency: 0, Cache Line Size: 64 bytes > Interrupt: pin A routed to IRQ 16 > Region 0: Memory at dcbe0000 (32-bit, non-prefetchable) [size=128K] > Region 2: I/O ports at bce0 [size=32] Region 3: Memory at dcbdc000 (32- > bit, non-prefetchable) [size=16K] > Capabilities: [c8] Power Management version 2 > Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2- > ,D3hot+,D3cold+) > Status: D0 PME-Enable- DSel=0 DScale=1 PME- > Capabilities: [d0] Message Signalled Interrupts: Mask- 64bit+ Count=1/1 > Enable- > Address: 0000000000000000 Data: 0000 > Capabilities: [e0] Express (v1) Endpoint, MSI 00 DevCap:MaxPayload 256 > bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- DevCtl:Report errors: > Correctable- Non-Fatal- Fatal- Unsupported- > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ > MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta:CorrErr+ UncorrErr- > FatalErr- UnsuppReq+ AuxPwr+ TransPend- LnkCap:Port #0, Speed 2.5GT/s, > Width x1, ASPM L0s L1, Latency L0 <128ns, L1 <64us > ClockPM- Suprise- LLActRep- BwNot- > LnkCtl:ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta:Speed 2.5GT/s, > Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- > ABWMgmt- > Capabilities: [a0] MSI-X: Enable+ Mask- TabSize=5 Vector table: BAR=3 > offset=00000000 > PBA: BAR=3 offset=00002000 > Capabilities: [100] Advanced Error Reporting > UESta:DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- > ECRC- UnsupReq- ACSVoil- > UEMsk:DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- > ECRC- UnsupReq- ACSVoil- > UESvrt:DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ > MalfTLP+ > ECRC- UnsupReq- ACSVoil- > CESta:RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ > CESta:RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ > AERCap:First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- > Capabilities: [140] Device Serial Number 84-4f-f0-ff-ff-9b-ca-70 Kernel > driver in use: e1000e Kernel modules: e1000e, e1000-cids > exec: /sbin/lspci -s 12:1.0 -vvv > 12:01.0 Class 0604: Device 10b5:8112 (rev aa) > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- DisINTx- > Status: Cap+ 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- > <TAbort- > <MAbort- >SERR- <PERR- INTx- > Latency: 32, Cache Line Size: 64 bytes > Region 0: Memory at dbcf0000 (64-bit, prefetchable) [size=64K] > Bus: primary=12, secondary=13, subordinate=16, sec-latency=32 I/O > behind bridge: 0000f000-00000fff Memory behind bridge: dc300000- > dc8fffff Prefetchable memory behind bridge: dbd00000-dc1fffff Secondary > status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ > <SERR- <PERR- > BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B- > PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- > Capabilities: [40] Power Management version 2 > Flags: PMEClk- DSI- D1+ D2- AuxCurrent=0mA PME(D0+,D1+,D2- > ,D3hot+,D3cold-) > Status: D0 PME-Enable- DSel=0 DScale=0 PME- > Capabilities: [50] Message Signalled Interrupts: Mask- 64bit+ Count=1/1 > Enable- > Address: 0000000000000000 Data: 0000 > Capabilities: [60] Express (v1) PCI/PCI-X to PCI-Express Bridge, MSI 00 > DevCap:MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us > ExtTag- RBE- FLReset- > DevCtl:Report errors: Correctable- Non-Fatal- Fatal- Unsupported- > RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- BrConfRtry- MaxPayload 128 > bytes, MaxReadReq 512 bytes DevSta:CorrErr+ UncorrErr- FatalErr- > UnsuppReq- AuxPwr- TransPend- LnkCap:Port #0, Speed 2.5GT/s, Width x1, > ASPM L0s L1, Latency L0 <1us, L1 <16us > ClockPM- Suprise- LLActRep- BwNot- > LnkCtl:ASPM Disabled; Disabled- Retrain- CommClk- > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta:Speed 2.5GT/s, > Width x1, TrErr- Train- SlotClk- DLActive- BWMgmt- > ABWMgmt- > CID Diagnostics Report END Mon May 20 21:58:51 UTC 2013 > > > > CLMBOH0001-IPS903# exit > > > On Tue, Jun 18, 2013 at 2:39 AM, Skidmore, Donald C < > [email protected]> wrote: > > > Hi Kaushal, > > > > Sorry about the troubles you're having with ixgbe. I was a bit > > confused by what exactly you're seeing so let me ask you a few > > questions. When you say " I see that out of 16 Rx Queues, only *One > > Queue* is dropping all the packets." is that queue getting the > > majority of the traffic or the same load as the other queues? I > would > > be interesting to see how your interrupt were laid out (cat > > /proc/interrupt | grep <ethX>) > > > > As far as why MPC incrementing this can happen due to two reasons. > > First we ran out of space in descriptor ring for the packet, this > > doesn't seem to be your case as you checked to make sure the ring was > > empty. Next we could be running on of bandwidth on the PCIe bus. > > This might be possible if all your traffic was going to just one > > queue, but it would be worthwhile to see what the connectivity looks > > like anyway (lspci -vvv) > > > > It is also worth noting that your driver is quite old 2.0.38.3. I > > would suggest at least trying the latest driver from source forge > > (currently 3.15.1). > > https://sourceforge.net/projects/e1000/files/ixgbe%20stable/ > > > > I would also like to see if anything is being logged while this > occurs > > (dmesg). > > > > Thanks, > > -Don Skidmore <[email protected]> > > > > > -----Original Message----- > > > From: Kaushal Bhandankar [mailto:[email protected]] > > > Sent: Sunday, June 16, 2013 12:32 PM > > > To: [email protected] > > > Subject: Re: [E1000-devel] Intel 82599 Driver dropping packets > Queue > > > Receive Descriptor Head > > > > > > -bash-3.2$ /home/service/ethtool_x64 -i po0_0 > > > driver: tun > > > version: 1.6 > > > firmware-version: N/A > > > bus-info: tap > > > supports-statistics: no > > > supports-test: no > > > supports-eeprom-access: no > > > supports-register-dump: no > > > > > > > > > -bash-3.2# modinfo ixgbe > > > filename: > /lib/modules/2.6.29.1/kernel/drivers/net/ixgbe/ixgbe.ko > > > version: 2.0.38.3-NAPI > > > license: GPL > > > description: Intel(R) 10 Gigabit PCI Express Network Driver > > > author: Intel Corporation, <[email protected]> > > > srcversion: CF158600678C3ADC41F341A > > > alias: pci:v00008086d000010FBsv*sd*bc*sc*i* > > > alias: pci:v00008086d000010FCsv*sd*bc*sc*i* > > > alias: pci:v00008086d000010F7sv*sd*bc*sc*i* > > > alias: pci:v00008086d000010DBsv*sd*bc*sc*i* > > > alias: pci:v00008086d000010F4sv*sd*bc*sc*i* > > > alias: pci:v00008086d000010E1sv*sd*bc*sc*i* > > > alias: pci:v00008086d000010F1sv*sd*bc*sc*i* > > > alias: pci:v00008086d000010ECsv*sd*bc*sc*i* > > > alias: pci:v00008086d000010DDsv*sd*bc*sc*i* > > > alias: pci:v00008086d0000150Bsv*sd*bc*sc*i* > > > alias: pci:v00008086d000010C8sv*sd*bc*sc*i* > > > alias: pci:v00008086d000010C7sv*sd*bc*sc*i* > > > alias: pci:v00008086d000010C6sv*sd*bc*sc*i* > > > alias: pci:v00008086d00001508sv*sd*bc*sc*i* > > > alias: pci:v00008086d000010B6sv*sd*bc*sc*i* > > > depends: > > > vermagic: 2.6.29.1 SMP mod_unload > > > parm: InterruptType:Change Interrupt Mode (0=Legacy, > 1=MSI, > > > 2=MSI-X), default 2 (array of int) > > > parm: MQ:Disable or enable Multiple Queues, default 1 > (array of > > > int) > > > parm: DCA:Disable or enable Direct Cache Access, > 0=disabled, > > > 1=descriptor only, 2=descriptor and data (array of int) > > > parm: RSS:Number of Receive-Side Scaling Descriptor > Queues, > > > default 1=number of cpus (array of int) > > > parm: InterruptThrottleRate:Maximum interrupts per > second, per > > > vector, (956-488281), default 8000 (array of int) > > > parm: LLIPort:Low Latency Interrupt TCP Port (0-65535) > (array > > of > > > int) > > > parm: LLIPush:Low Latency Interrupt on TCP Push flag > (0,1) > > (array > > > of int) > > > parm: LLISize:Low Latency Interrupt on Packet Size (0- > 1500) > > > (array of int) > > > parm: LLIEType:Low Latency Interrupt Ethernet Protocol > Type > > > (array of int) > > > parm: LLIVLANP:Low Latency Interrupt on VLAN priority > threshold > > > (array of int) > > > parm: RxBufferMode:0=1 descriptor per packet, > > > 1=use packet split, multiple descriptors > per > > jumbo frame > > > 2 (default)=use 1buf mode for 1500 mtu, > > > packet > > split for jumbo > > > (array of int) > > > parm: FdirMode:Flow Director filtering modes: > > > 0 = Filtering off > > > 1 = Signature Hashing filters (SW ATR) > > > 2 = Perfect Filters (array of int) > > > parm: FdirPballoc:Flow Director packet buffer allocation > level: > > > 0 = 8k hash filters or 2k perfect filters > > > 1 = 16k hash filters or 4k perfect filters > > > 2 = 32k hash filters or 8k perfect filters > > (array of int) > > > parm: AtrSampleRate:Software ATR Tx packet sample rate > (array > > of > > > int) > > > parm: DoubleVlan:Disable or enable double Vlan support, > > default 0 > > > (array of int) > > > parm: InnerVlanMode:Disable or enable Inner Vlan > stripping, > > > default 0 (array of int) > > > > > > > > > > > > -bash-3.2# modprobe -l ixgbe > > > /lib/modules/2.6.29.1/kernel/drivers/net/ixgbe/ixgbe.ko > > > > > > -bash-3.2# lsmod > > > Module Size Used by > > > cidmodcap 4208 16 > > > cpp_base 845040 8 > > > tipc 118392 2 > > > rebootkom 2468 0 > > > nf_conntrack_ipv4 13376 1 > > > nf_defrag_ipv4 1976 1 nf_conntrack_ipv4 > > > xt_state 2232 1 > > > nf_conntrack 60000 2 nf_conntrack_ipv4,xt_state > > > iptable_filter 2888 1 > > > ip_tables 15848 1 iptable_filter > > > x_tables 18208 2 xt_state,ip_tables > > > igb 76876 0 > > > e1000e 113256 0 > > > ixgbe 168116 0 > > > ihm 6252 2 > > > cids_shared 579704 0 > > > linux_user_bde 15624 0 > > > linux_kernel_bde 28816 1 linux_user_bde > > > -bash-3.2# > > > > > > > > > cat /proc/cpuinfo = 16 cores > > > > > > > > > > > > > > > On Sun, Jun 16, 2013 at 10:04 AM, Kaushal Bhandankar > > > <[email protected]>wrote: > > > > > > > Hi, > > > > I am using Intel 82599 driver in my product. I see that out of 16 > > > > Rx Queues, only *One Queue* is dropping all the packets. When I > > > > did a test *to en-queue packets only to the problematic Queue*, I > > > > found that the > > > > > > > > -> Rx Missed Packet Count ( mpc ) is 0 Good Packet Received Count > > > > -> ( gprc ) in increased for received packets - > > > > L2 filtering is pass. > > > > > > > > What may be the reason for this behavior ? > > > > > > > > For debug, I am also printing the "Queue Receive Descriptor Head" > > > > and "Queue Receive Descriptor Tail" to get information about how > > > > many descriptors are in-use. My question is, in further tests, if > > > > I find that No Descriptor is in-use, however still packets are > > > > getting dropped -- Can I infer Hardware Failure from it ? > > > > > > > > Regards, > > > > Kaushal > > > > > > ------------------------------------------------------------------------------ This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev _______________________________________________ E1000-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
