Hey Lukasz, It's a HW limit the RSS hash on this device only returns 4bits so we only can mux to at most 16 queues. Later HW as more bits and thus can spread RSS over more queues.
Thanks, -Don > -----Original Message----- > From: Łukasz Chrustek [mailto:luk...@chrustek.net] > Sent: Thursday, October 06, 2016 1:22 PM > To: Skidmore, Donald C <donald.c.skidm...@intel.com>; Tal Abudi > <talab...@gmail.com> > Cc: e1000-devel@lists.sourceforge.net > Subject: Re: [E1000-devel] rx queue > > Hello Don, > > So if it defaults RSS, is there any chance to make it bigger then > 16 ? Is it driver or hardware limitation ? > > Thank You for clarification of this case ! > > Regards > Lukasz > > > Hey Lukasz, > > > ATR work by targeting TCP flows as the adapter transmits a SYN packet. > > At this point it creates a new hash bucket with the intent of > > redirecting other traffic of this flow to the same queue was > > transmitted on. The idea being this what the received packets will > > arrive at the same CPU as the transmitted. Clearly there is lots of > > traffic this wouldn't work well with (i.e. most packet generators, > > short lived connections, UPD). So if no hash is found for Rx traffic > > we default to RSS, which as I mentioned mux's over only the first 16 > > queues due to the hash limitation. > > > I believe this is what you are seeing. This is reinforced by the > > number packets that are missing the ATR hash: > > > fdir_match: 107665723 > > fdir_miss: 31074999856 > > > Thanks, > > -Don Skidmore <donald.c.skidm...@intel.com> > > > >> -----Original Message----- > >> From: Łukasz Chrustek [mailto:luk...@chrustek.net] > >> Sent: Thursday, October 06, 2016 12:09 PM > >> To: Skidmore, Donald C <donald.c.skidm...@intel.com>; Tal Abudi > >> <talab...@gmail.com> > >> Cc: e1000-devel@lists.sourceforge.net > >> Subject: Re: [E1000-devel] rx queue > >> > >> Hello, > >> > >> I didn't change any default config, so if ATR is turned on by > >> default then it is on. Flows - this was generated flood traffic from > >> command > >> hping3 -2 --flod --rand-source <target IP>. <target IP> is > >> behind problematic server - the server is router. Can You tell what > >> is the way to verify this by ethtool ? > >> > >> Regards > >> Lukasz > >> > >> > It looks like your just using RSS to spread most of your traffic > >> > between queues who's hash is limited to 4 bits (16 queues). This > >> > would happen if you turned off ATR or your flows weren't applicably > >> > for ATR. You should be able to verify this by the ethtool statistic. > >> > >> > Thanks, > >> > -Don Skidmore <donald.c.skidm...@intel.com> > >> > >> > >> >> -----Original Message----- > >> >> From: Łukasz Chrustek [mailto:luk...@chrustek.net] > >> >> Sent: Thursday, October 06, 2016 8:23 AM > >> >> To: Tal Abudi <talab...@gmail.com> > >> >> Cc: e1000-devel@lists.sourceforge.net > >> >> Subject: Re: [E1000-devel] rx queue > >> >> > >> >> Witam, > >> >> > >> >> > Hi > >> >> > What type of system are you running ? > >> >> > >> >> it is ubuntu 16.04 LTS > >> >> > >> >> > Which ethernet controller (lspci -i net) and driver (ethtool -i > >> >> > S8P0) > >> >> # lspci -i net > >> >> 00:00.0 Host bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon > >> >> E3 v4/Xeon D DMI2 (rev 01) > >> >> 00:01.0 PCI bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon > >> >> E3 v4/Xeon D PCI Express Root Port 1 (rev 01) > >> >> 00:02.0 PCI bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon > >> >> E3 v4/Xeon D PCI Express Root Port 2 (rev 01) > >> >> 00:03.0 PCI bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon > >> >> E3 v4/Xeon D PCI Express Root Port 3 (rev 01) > >> >> 00:03.2 PCI bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon > >> >> E3 v4/Xeon D PCI Express Root Port 3 (rev 01) > >> >> 00:04.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 0 (rev 01) > >> >> 00:04.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 1 (rev 01) > >> >> 00:04.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 2 (rev 01) > >> >> 00:04.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 3 (rev 01) > >> >> 00:04.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 4 (rev 01) > >> >> 00:04.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 5 (rev 01) > >> >> 00:04.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 6 (rev 01) > >> >> 00:04.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 7 (rev 01) > >> >> 00:05.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Map/VTd_Misc/System Management (rev 01) > >> >> 00:05.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D IIO Hot Plug (rev 01) > >> >> 00:05.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D IIO RAS/Control Status/Global Errors (rev 01) > >> >> 00:05.4 PIC: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 > >> >> v4/Xeon D I/O APIC (rev 01) > >> >> 00:11.0 Unassigned class [ff00]: Intel Corporation C610/X99 series > >> >> chipset SPSR (rev 05) > >> >> 00:11.4 SATA controller: Intel Corporation C610/X99 series chipset > >> >> sSATA Controller [AHCI mode] (rev 05) > >> >> 00:14.0 USB controller: Intel Corporation C610/X99 series chipset > >> >> USB xHCI Host Controller (rev 05) > >> >> 00:16.0 Communication controller: Intel Corporation C610/X99 > >> >> series chipset MEI Controller (rev 05) > >> >> 00:16.1 Communication controller: Intel Corporation C610/X99 > >> >> series chipset MEI Controller (rev 05) > >> >> 00:1a.0 USB controller: Intel Corporation C610/X99 series chipset > >> >> USB Enhanced Host Controller (rev 05) > >> >> 00:1c.0 PCI bridge: Intel Corporation C610/X99 series chipset PCI > >> >> Express Root Port (rev d5) > >> >> 00:1c.4 PCI bridge: Intel Corporation C610/X99 series chipset PCI > >> >> Express Root Port (rev d5) > >> >> 00:1d.0 USB controller: Intel Corporation C610/X99 series chipset > >> >> USB Enhanced Host Controller (rev 05) > >> >> 00:1f.0 ISA bridge: Intel Corporation C610/X99 series chipset LPC > >> >> Controller (rev 05) > >> >> 00:1f.2 SATA controller: Intel Corporation C610/X99 series chipset > >> >> 6-Port SATA Controller [AHCI mode] (rev 05) > >> >> 00:1f.3 SMBus: Intel Corporation C610/X99 series chipset SMBus > >> >> Controller (rev 05) > >> >> 01:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit > >> >> SFI/SFP+ Network Connection (rev 01) > >> >> 01:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit > >> >> SFI/SFP+ Network Connection (rev 01) > >> >> 02:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit > >> >> SFI/SFP+ Network Connection (rev 01) > >> >> 02:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit > >> >> SFI/SFP+ Network Connection (rev 01) > >> >> 04:00.0 Ethernet controller: Intel Corporation I350 Gigabit > >> >> Network Connection (rev 01) > >> >> 04:00.1 Ethernet controller: Intel Corporation I350 Gigabit > >> >> Network Connection (rev 01) > >> >> 06:00.0 PCI bridge: ASPEED Technology, Inc. AST1150 PCI-to-PCI > >> >> Bridge (rev > >> >> 03) > >> >> 07:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED > >> >> Graphics Family (rev 30) > >> >> 7f:08.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D QPI Link 0 (rev 01) > >> >> 7f:08.2 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon > >> >> E3 v4/Xeon D QPI Link 0 (rev 01) > >> >> 7f:08.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D QPI Link 0 (rev 01) > >> >> 7f:09.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D QPI Link 1 (rev 01) > >> >> 7f:09.2 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon > >> >> E3 v4/Xeon D QPI Link 1 (rev 01) > >> >> 7f:09.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D QPI Link 1 (rev 01) > >> >> 7f:0b.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D R3 QPI Link 0/1 (rev 01) > >> >> 7f:0b.1 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon > >> >> E3 v4/Xeon D R3 QPI Link 0/1 (rev 01) > >> >> 7f:0b.2 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon > >> >> E3 v4/Xeon D R3 QPI Link 0/1 (rev 01) > >> >> 7f:0b.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D R3 QPI Link Debug (rev 01) > >> >> 7f:0c.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> 7f:0c.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> 7f:0c.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> 7f:0c.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> 7f:0c.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> 7f:0c.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> 7f:0c.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> 7f:0c.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> 7f:0d.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> 7f:0d.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> 7f:0d.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> 7f:0d.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> 7f:0f.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> 7f:0f.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> 7f:0f.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> 7f:0f.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> 7f:0f.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> 7f:0f.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> 7f:0f.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> 7f:10.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D R2PCIe Agent (rev 01) > >> >> 7f:10.1 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon > >> >> E3 v4/Xeon D R2PCIe Agent (rev 01) > >> >> 7f:10.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Ubox (rev 01) > >> >> 7f:10.6 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon > >> >> E3 v4/Xeon D Ubox (rev 01) > >> >> 7f:10.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Ubox (rev 01) > >> >> 7f:12.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Home Agent 0 (rev 01) > >> >> 7f:12.1 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon > >> >> E3 v4/Xeon D Home Agent 0 (rev 01) > >> >> 7f:12.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Home Agent 1 (rev 01) > >> >> 7f:12.5 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon > >> >> E3 v4/Xeon D Home Agent 1 (rev 01) > >> >> 7f:13.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Target > >> >> Address/Thermal/RAS (rev 01) > >> >> 7f:13.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Target > >> >> Address/Thermal/RAS (rev 01) > >> >> 7f:13.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel Target Address > >> >> Decoder (rev 01) > >> >> 7f:13.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel Target Address > >> >> Decoder (rev 01) > >> >> 7f:13.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Broadcast (rev 01) > >> >> 7f:13.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Global Broadcast (rev 01) > >> >> 7f:14.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 0 Thermal > >> >> Control (rev 01) > >> >> 7f:14.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 1 Thermal > >> >> Control (rev 01) > >> >> 7f:14.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 0 Error (rev > >> >> 01) > >> >> 7f:14.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 1 Error (rev > >> >> 01) > >> >> 7f:14.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface (rev 01) > >> >> 7f:14.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface (rev 01) > >> >> 7f:14.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface (rev 01) > >> >> 7f:14.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface (rev 01) > >> >> 7f:16.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Target Address/Thermal/RAS (rev 01) > >> >> 7f:16.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Target Address/Thermal/RAS (rev 01) > >> >> 7f:16.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Channel Target Address Decoder (rev 01) > >> >> 7f:16.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Channel Target Address Decoder (rev 01) > >> >> 7f:16.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Broadcast (rev 01) > >> >> 7f:16.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Global Broadcast (rev 01) > >> >> 7f:17.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 1 - Channel 0 Thermal > >> >> Control (rev 01) > >> >> 7f:17.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 1 - Channel 1 Thermal > >> >> Control (rev 01) > >> >> 7f:17.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 1 - Channel 0 Error (rev > >> >> 01) > >> >> 7f:17.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 1 - Channel 1 Error (rev > >> >> 01) > >> >> 7f:17.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface (rev 01) > >> >> 7f:17.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface (rev 01) > >> >> 7f:17.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface (rev 01) > >> >> 7f:17.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface (rev 01) > >> >> 7f:1e.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) > >> >> 7f:1e.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) > >> >> 7f:1e.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) > >> >> 7f:1e.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) > >> >> 7f:1e.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) > >> >> 7f:1f.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) > >> >> 7f:1f.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) > >> >> 80:01.0 PCI bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon > >> >> E3 v4/Xeon D PCI Express Root Port 1 (rev 01) > >> >> 80:03.0 PCI bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon > >> >> E3 v4/Xeon D PCI Express Root Port 3 (rev 01) > >> >> 80:04.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 0 (rev 01) > >> >> 80:04.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 1 (rev 01) > >> >> 80:04.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 2 (rev 01) > >> >> 80:04.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 3 (rev 01) > >> >> 80:04.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 4 (rev 01) > >> >> 80:04.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 5 (rev 01) > >> >> 80:04.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 6 (rev 01) > >> >> 80:04.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 7 (rev 01) > >> >> 80:05.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Map/VTd_Misc/System Management (rev 01) > >> >> 80:05.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D IIO Hot Plug (rev 01) > >> >> 80:05.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D IIO RAS/Control Status/Global Errors (rev 01) > >> >> 80:05.4 PIC: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 > >> >> v4/Xeon D I/O APIC (rev 01) > >> >> 81:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit > >> >> SFI/SFP+ Network Connection (rev 01) > >> >> 81:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit > >> >> SFI/SFP+ Network Connection (rev 01) > >> >> 82:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit > >> >> SFI/SFP+ Network Connection (rev 01) > >> >> 82:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit > >> >> SFI/SFP+ Network Connection (rev 01) > >> >> ff:08.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D QPI Link 0 (rev 01) > >> >> ff:08.2 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon > >> >> E3 v4/Xeon D QPI Link 0 (rev 01) > >> >> ff:08.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D QPI Link 0 (rev 01) > >> >> ff:09.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D QPI Link 1 (rev 01) > >> >> ff:09.2 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon > >> >> E3 v4/Xeon D QPI Link 1 (rev 01) > >> >> ff:09.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D QPI Link 1 (rev 01) > >> >> ff:0b.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D R3 QPI Link 0/1 (rev 01) > >> >> ff:0b.1 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon > >> >> E3 v4/Xeon D R3 QPI Link 0/1 (rev 01) > >> >> ff:0b.2 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon > >> >> E3 v4/Xeon D R3 QPI Link 0/1 (rev 01) > >> >> ff:0b.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D R3 QPI Link Debug (rev 01) > >> >> ff:0c.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> ff:0c.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> ff:0c.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> ff:0c.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> ff:0c.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> ff:0c.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> ff:0c.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> ff:0c.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> ff:0d.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> ff:0d.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> ff:0d.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> ff:0d.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> ff:0f.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> ff:0f.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> ff:0f.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> ff:0f.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> ff:0f.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> ff:0f.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> ff:0f.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) > >> >> ff:10.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D R2PCIe Agent (rev 01) > >> >> ff:10.1 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon > >> >> E3 v4/Xeon D R2PCIe Agent (rev 01) > >> >> ff:10.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Ubox (rev 01) > >> >> ff:10.6 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon > >> >> E3 v4/Xeon D Ubox (rev 01) > >> >> ff:10.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Ubox (rev 01) > >> >> ff:12.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Home Agent 0 (rev 01) > >> >> ff:12.1 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon > >> >> E3 v4/Xeon D Home Agent 0 (rev 01) > >> >> ff:12.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Home Agent 1 (rev 01) > >> >> ff:12.5 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon > >> >> E3 v4/Xeon D Home Agent 1 (rev 01) > >> >> ff:13.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Target > >> >> Address/Thermal/RAS (rev 01) > >> >> ff:13.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Target > >> >> Address/Thermal/RAS (rev 01) > >> >> ff:13.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel Target Address > >> >> Decoder (rev 01) > >> >> ff:13.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel Target Address > >> >> Decoder (rev 01) > >> >> ff:13.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Broadcast (rev 01) > >> >> ff:13.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Global Broadcast (rev 01) > >> >> ff:14.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 0 Thermal > >> >> Control (rev 01) > >> >> ff:14.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 1 Thermal > >> >> Control (rev 01) > >> >> ff:14.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 0 Error (rev > >> >> 01) > >> >> ff:14.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 1 Error (rev > >> >> 01) > >> >> ff:14.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface (rev 01) > >> >> ff:14.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface (rev 01) > >> >> ff:14.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface (rev 01) > >> >> ff:14.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface (rev 01) > >> >> ff:16.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Target Address/Thermal/RAS (rev 01) > >> >> ff:16.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Target Address/Thermal/RAS (rev 01) > >> >> ff:16.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Channel Target Address Decoder (rev 01) > >> >> ff:16.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Channel Target Address Decoder (rev 01) > >> >> ff:16.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Broadcast (rev 01) > >> >> ff:16.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Global Broadcast (rev 01) > >> >> ff:17.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 1 - Channel 0 Thermal > >> >> Control (rev 01) > >> >> ff:17.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 1 - Channel 1 Thermal > >> >> Control (rev 01) > >> >> ff:17.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 1 - Channel 0 Error (rev > >> >> 01) > >> >> ff:17.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Memory Controller 1 - Channel 1 Error (rev > >> >> 01) > >> >> ff:17.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface (rev 01) > >> >> ff:17.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface (rev 01) > >> >> ff:17.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface (rev 01) > >> >> ff:17.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface (rev 01) > >> >> ff:1e.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) > >> >> ff:1e.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) > >> >> ff:1e.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) > >> >> ff:1e.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) > >> >> ff:1e.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) > >> >> ff:1f.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) > >> >> ff:1f.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 > >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) > >> >> > >> >> > >> >> # ethtool -i S8P0 > >> >> driver: ixgbe > >> >> version: 4.4.6 > >> >> firmware-version: 0x61c10001 > >> >> expansion-rom-version: > >> >> bus-info: 0000:82:00.0 > >> >> supports-statistics: yes > >> >> supports-test: yes > >> >> supports-eeprom-access: yes > >> >> supports-register-dump: yes > >> >> supports-priv-flags: yes > >> >> > >> >> Regards > >> >> Lukasz > >> >> > >> >> > >> >> > Thanks, > >> >> > Tal > >> >> > >> >> > On Thu, Oct 6, 2016 at 1:06 PM, Łukasz Chrustek > >> >> > <luk...@chrustek.net> > >> >> wrote: > >> >> > >> >> > Hello, > >> >> > >> >> > I have strange problem, my system has two 12-core > >> >> > procesors, when setting smp_affinity and rps_cpus to spread > >> >> > load accros all cores, I get following: > >> >> > >> >> > ethtool -S S8P0 |grep -E "rx.*pack" > >> >> > rx_packets: 28059349668 > >> >> > rx_fcoe_packets: 0 > >> >> > rx_queue_0_packets: 1739671933 > >> >> > rx_queue_1_packets: 1740406739 > >> >> > rx_queue_2_packets: 1945709824 > >> >> > rx_queue_3_packets: 1741279872 > >> >> > rx_queue_4_packets: 1739593471 > >> >> > rx_queue_5_packets: 1740392727 > >> >> > rx_queue_6_packets: 1740513001 > >> >> > rx_queue_7_packets: 1740921491 > >> >> > rx_queue_8_packets: 1741525459 > >> >> > rx_queue_9_packets: 1739646341 > >> >> > rx_queue_10_packets: 1741068593 > >> >> > rx_queue_11_packets: 1741374592 > >> >> > rx_queue_12_packets: 1739709430 > >> >> > rx_queue_13_packets: 1747586478 > >> >> > rx_queue_14_packets: 1740398506 > >> >> > rx_queue_15_packets: 1739546993 > >> >> > rx_queue_16_packets: 690 > >> >> > rx_queue_17_packets: 455 > >> >> > rx_queue_18_packets: 653 > >> >> > rx_queue_19_packets: 411 > >> >> > rx_queue_20_packets: 510 > >> >> > rx_queue_21_packets: 514 > >> >> > rx_queue_22_packets: 588 > >> >> > rx_queue_23_packets: 431 > >> >> > >> >> > >> >> > but for tx: > >> >> > >> >> > >> >> > ethtool -S S8P0 |grep -E "tx.*pack" > >> >> > tx_packets: 23697264990 > >> >> > tx_fcoe_packets: 0 > >> >> > tx_queue_0_packets: 897068903 > >> >> > tx_queue_1_packets: 897204633 > >> >> > tx_queue_2_packets: 1705630673 > >> >> > tx_queue_3_packets: 897124425 > >> >> > tx_queue_4_packets: 897124753 > >> >> > tx_queue_5_packets: 897114279 > >> >> > tx_queue_6_packets: 897379101 > >> >> > tx_queue_7_packets: 897218157 > >> >> > tx_queue_8_packets: 904811070 > >> >> > tx_queue_9_packets: 897124394 > >> >> > tx_queue_10_packets: 901837638 > >> >> > tx_queue_11_packets: 897419500 > >> >> > tx_queue_12_packets: 1241034979 > >> >> > tx_queue_13_packets: 1247080319 > >> >> > tx_queue_14_packets: 1247499026 > >> >> > tx_queue_15_packets: 1240739566 > >> >> > tx_queue_16_packets: 891720194 > >> >> > tx_queue_17_packets: 891610281 > >> >> > tx_queue_18_packets: 891814848 > >> >> > tx_queue_19_packets: 891770115 > >> >> > tx_queue_20_packets: 891746387 > >> >> > tx_queue_21_packets: 891680822 > >> >> > tx_queue_22_packets: 891775521 > >> >> > tx_queue_23_packets: 891735428 > >> >> > >> >> > >> >> > Can You provide some clue to spread rx_queue for all cores > >> >> > not only first 16 ? > >> >> > > > > > -- > Pozdrawiam, > Łukasz Chrustek ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
