Hello Don, Can You share what HW You suggest ? Which chipset from intel portfolio ? More queues can use more cores, so it is simple choice for me - I need newer HW :)
Regards Lukasz > Hey Lukasz, > It's a HW limit the RSS hash on this device only returns 4bits so > we only can mux to at most 16 queues. Later HW as more bits and > thus can spread RSS over more queues. > Thanks, > -Don >> -----Original Message----- >> From: Łukasz Chrustek [mailto:luk...@chrustek.net] >> Sent: Thursday, October 06, 2016 1:22 PM >> To: Skidmore, Donald C <donald.c.skidm...@intel.com>; Tal Abudi >> <talab...@gmail.com> >> Cc: e1000-devel@lists.sourceforge.net >> Subject: Re: [E1000-devel] rx queue >> >> Hello Don, >> >> So if it defaults RSS, is there any chance to make it bigger then >> 16 ? Is it driver or hardware limitation ? >> >> Thank You for clarification of this case ! >> >> Regards >> Lukasz >> >> > Hey Lukasz, >> >> > ATR work by targeting TCP flows as the adapter transmits a SYN packet. >> > At this point it creates a new hash bucket with the intent of >> > redirecting other traffic of this flow to the same queue was >> > transmitted on. The idea being this what the received packets will >> > arrive at the same CPU as the transmitted. Clearly there is lots of >> > traffic this wouldn't work well with (i.e. most packet generators, >> > short lived connections, UPD). So if no hash is found for Rx traffic >> > we default to RSS, which as I mentioned mux's over only the first 16 >> > queues due to the hash limitation. >> >> > I believe this is what you are seeing. This is reinforced by the >> > number packets that are missing the ATR hash: >> >> > fdir_match: 107665723 >> > fdir_miss: 31074999856 >> >> > Thanks, >> > -Don Skidmore <donald.c.skidm...@intel.com> >> >> >> >> -----Original Message----- >> >> From: Łukasz Chrustek [mailto:luk...@chrustek.net] >> >> Sent: Thursday, October 06, 2016 12:09 PM >> >> To: Skidmore, Donald C <donald.c.skidm...@intel.com>; Tal Abudi >> >> <talab...@gmail.com> >> >> Cc: e1000-devel@lists.sourceforge.net >> >> Subject: Re: [E1000-devel] rx queue >> >> >> >> Hello, >> >> >> >> I didn't change any default config, so if ATR is turned on by >> >> default then it is on. Flows - this was generated flood traffic from >> >> command >> >> hping3 -2 --flod --rand-source <target IP>. <target IP> is >> >> behind problematic server - the server is router. Can You tell what >> >> is the way to verify this by ethtool ? >> >> >> >> Regards >> >> Lukasz >> >> >> >> > It looks like your just using RSS to spread most of your traffic >> >> > between queues who's hash is limited to 4 bits (16 queues). This >> >> > would happen if you turned off ATR or your flows weren't applicably >> >> > for ATR. You should be able to verify this by the ethtool statistic. >> >> >> >> > Thanks, >> >> > -Don Skidmore <donald.c.skidm...@intel.com> >> >> >> >> >> >> >> -----Original Message----- >> >> >> From: Łukasz Chrustek [mailto:luk...@chrustek.net] >> >> >> Sent: Thursday, October 06, 2016 8:23 AM >> >> >> To: Tal Abudi <talab...@gmail.com> >> >> >> Cc: e1000-devel@lists.sourceforge.net >> >> >> Subject: Re: [E1000-devel] rx queue >> >> >> >> >> >> Witam, >> >> >> >> >> >> > Hi >> >> >> > What type of system are you running ? >> >> >> >> >> >> it is ubuntu 16.04 LTS >> >> >> >> >> >> > Which ethernet controller (lspci -i net) and driver (ethtool -i >> >> >> > S8P0) >> >> >> # lspci -i net >> >> >> 00:00.0 Host bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon >> >> >> E3 v4/Xeon D DMI2 (rev 01) >> >> >> 00:01.0 PCI bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon >> >> >> E3 v4/Xeon D PCI Express Root Port 1 (rev 01) >> >> >> 00:02.0 PCI bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon >> >> >> E3 v4/Xeon D PCI Express Root Port 2 (rev 01) >> >> >> 00:03.0 PCI bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon >> >> >> E3 v4/Xeon D PCI Express Root Port 3 (rev 01) >> >> >> 00:03.2 PCI bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon >> >> >> E3 v4/Xeon D PCI Express Root Port 3 (rev 01) >> >> >> 00:04.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 0 (rev 01) >> >> >> 00:04.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 1 (rev 01) >> >> >> 00:04.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 2 (rev 01) >> >> >> 00:04.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 3 (rev 01) >> >> >> 00:04.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 4 (rev 01) >> >> >> 00:04.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 5 (rev 01) >> >> >> 00:04.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 6 (rev 01) >> >> >> 00:04.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 7 (rev 01) >> >> >> 00:05.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Map/VTd_Misc/System Management (rev 01) >> >> >> 00:05.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D IIO Hot Plug (rev 01) >> >> >> 00:05.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D IIO RAS/Control Status/Global Errors (rev 01) >> >> >> 00:05.4 PIC: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 >> >> >> v4/Xeon D I/O APIC (rev 01) >> >> >> 00:11.0 Unassigned class [ff00]: Intel Corporation C610/X99 series >> >> >> chipset SPSR (rev 05) >> >> >> 00:11.4 SATA controller: Intel Corporation C610/X99 series chipset >> >> >> sSATA Controller [AHCI mode] (rev 05) >> >> >> 00:14.0 USB controller: Intel Corporation C610/X99 series chipset >> >> >> USB xHCI Host Controller (rev 05) >> >> >> 00:16.0 Communication controller: Intel Corporation C610/X99 >> >> >> series chipset MEI Controller (rev 05) >> >> >> 00:16.1 Communication controller: Intel Corporation C610/X99 >> >> >> series chipset MEI Controller (rev 05) >> >> >> 00:1a.0 USB controller: Intel Corporation C610/X99 series chipset >> >> >> USB Enhanced Host Controller (rev 05) >> >> >> 00:1c.0 PCI bridge: Intel Corporation C610/X99 series chipset PCI >> >> >> Express Root Port (rev d5) >> >> >> 00:1c.4 PCI bridge: Intel Corporation C610/X99 series chipset PCI >> >> >> Express Root Port (rev d5) >> >> >> 00:1d.0 USB controller: Intel Corporation C610/X99 series chipset >> >> >> USB Enhanced Host Controller (rev 05) >> >> >> 00:1f.0 ISA bridge: Intel Corporation C610/X99 series chipset LPC >> >> >> Controller (rev 05) >> >> >> 00:1f.2 SATA controller: Intel Corporation C610/X99 series chipset >> >> >> 6-Port SATA Controller [AHCI mode] (rev 05) >> >> >> 00:1f.3 SMBus: Intel Corporation C610/X99 series chipset SMBus >> >> >> Controller (rev 05) >> >> >> 01:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit >> >> >> SFI/SFP+ Network Connection (rev 01) >> >> >> 01:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit >> >> >> SFI/SFP+ Network Connection (rev 01) >> >> >> 02:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit >> >> >> SFI/SFP+ Network Connection (rev 01) >> >> >> 02:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit >> >> >> SFI/SFP+ Network Connection (rev 01) >> >> >> 04:00.0 Ethernet controller: Intel Corporation I350 Gigabit >> >> >> Network Connection (rev 01) >> >> >> 04:00.1 Ethernet controller: Intel Corporation I350 Gigabit >> >> >> Network Connection (rev 01) >> >> >> 06:00.0 PCI bridge: ASPEED Technology, Inc. AST1150 PCI-to-PCI >> >> >> Bridge (rev >> >> >> 03) >> >> >> 07:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED >> >> >> Graphics Family (rev 30) >> >> >> 7f:08.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D QPI Link 0 (rev 01) >> >> >> 7f:08.2 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon >> >> >> E3 v4/Xeon D QPI Link 0 (rev 01) >> >> >> 7f:08.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D QPI Link 0 (rev 01) >> >> >> 7f:09.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D QPI Link 1 (rev 01) >> >> >> 7f:09.2 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon >> >> >> E3 v4/Xeon D QPI Link 1 (rev 01) >> >> >> 7f:09.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D QPI Link 1 (rev 01) >> >> >> 7f:0b.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D R3 QPI Link 0/1 (rev 01) >> >> >> 7f:0b.1 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon >> >> >> E3 v4/Xeon D R3 QPI Link 0/1 (rev 01) >> >> >> 7f:0b.2 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon >> >> >> E3 v4/Xeon D R3 QPI Link 0/1 (rev 01) >> >> >> 7f:0b.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D R3 QPI Link Debug (rev 01) >> >> >> 7f:0c.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> 7f:0c.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> 7f:0c.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> 7f:0c.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> 7f:0c.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> 7f:0c.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> 7f:0c.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> 7f:0c.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> 7f:0d.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> 7f:0d.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> 7f:0d.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> 7f:0d.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> 7f:0f.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> 7f:0f.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> 7f:0f.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> 7f:0f.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> 7f:0f.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> 7f:0f.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> 7f:0f.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> 7f:10.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D R2PCIe Agent (rev 01) >> >> >> 7f:10.1 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon >> >> >> E3 v4/Xeon D R2PCIe Agent (rev 01) >> >> >> 7f:10.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Ubox (rev 01) >> >> >> 7f:10.6 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon >> >> >> E3 v4/Xeon D Ubox (rev 01) >> >> >> 7f:10.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Ubox (rev 01) >> >> >> 7f:12.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Home Agent 0 (rev 01) >> >> >> 7f:12.1 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon >> >> >> E3 v4/Xeon D Home Agent 0 (rev 01) >> >> >> 7f:12.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Home Agent 1 (rev 01) >> >> >> 7f:12.5 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon >> >> >> E3 v4/Xeon D Home Agent 1 (rev 01) >> >> >> 7f:13.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Target >> >> >> Address/Thermal/RAS (rev 01) >> >> >> 7f:13.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Target >> >> >> Address/Thermal/RAS (rev 01) >> >> >> 7f:13.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel Target Address >> >> >> Decoder (rev 01) >> >> >> 7f:13.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel Target Address >> >> >> Decoder (rev 01) >> >> >> 7f:13.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Broadcast (rev 01) >> >> >> 7f:13.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Global Broadcast (rev 01) >> >> >> 7f:14.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 0 Thermal >> >> >> Control (rev 01) >> >> >> 7f:14.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 1 Thermal >> >> >> Control (rev 01) >> >> >> 7f:14.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 0 Error (rev >> >> >> 01) >> >> >> 7f:14.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 1 Error (rev >> >> >> 01) >> >> >> 7f:14.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface (rev 01) >> >> >> 7f:14.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface (rev 01) >> >> >> 7f:14.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface (rev 01) >> >> >> 7f:14.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface (rev 01) >> >> >> 7f:16.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Target Address/Thermal/RAS (rev 01) >> >> >> 7f:16.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Target Address/Thermal/RAS (rev 01) >> >> >> 7f:16.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Channel Target Address Decoder (rev 01) >> >> >> 7f:16.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Channel Target Address Decoder (rev 01) >> >> >> 7f:16.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Broadcast (rev 01) >> >> >> 7f:16.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Global Broadcast (rev 01) >> >> >> 7f:17.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 1 - Channel 0 Thermal >> >> >> Control (rev 01) >> >> >> 7f:17.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 1 - Channel 1 Thermal >> >> >> Control (rev 01) >> >> >> 7f:17.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 1 - Channel 0 Error (rev >> >> >> 01) >> >> >> 7f:17.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 1 - Channel 1 Error (rev >> >> >> 01) >> >> >> 7f:17.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface (rev 01) >> >> >> 7f:17.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface (rev 01) >> >> >> 7f:17.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface (rev 01) >> >> >> 7f:17.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface (rev 01) >> >> >> 7f:1e.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) >> >> >> 7f:1e.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) >> >> >> 7f:1e.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) >> >> >> 7f:1e.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) >> >> >> 7f:1e.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) >> >> >> 7f:1f.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) >> >> >> 7f:1f.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) >> >> >> 80:01.0 PCI bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon >> >> >> E3 v4/Xeon D PCI Express Root Port 1 (rev 01) >> >> >> 80:03.0 PCI bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon >> >> >> E3 v4/Xeon D PCI Express Root Port 3 (rev 01) >> >> >> 80:04.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 0 (rev 01) >> >> >> 80:04.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 1 (rev 01) >> >> >> 80:04.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 2 (rev 01) >> >> >> 80:04.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 3 (rev 01) >> >> >> 80:04.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 4 (rev 01) >> >> >> 80:04.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 5 (rev 01) >> >> >> 80:04.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 6 (rev 01) >> >> >> 80:04.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 7 (rev 01) >> >> >> 80:05.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Map/VTd_Misc/System Management (rev 01) >> >> >> 80:05.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D IIO Hot Plug (rev 01) >> >> >> 80:05.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D IIO RAS/Control Status/Global Errors (rev 01) >> >> >> 80:05.4 PIC: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 >> >> >> v4/Xeon D I/O APIC (rev 01) >> >> >> 81:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit >> >> >> SFI/SFP+ Network Connection (rev 01) >> >> >> 81:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit >> >> >> SFI/SFP+ Network Connection (rev 01) >> >> >> 82:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit >> >> >> SFI/SFP+ Network Connection (rev 01) >> >> >> 82:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit >> >> >> SFI/SFP+ Network Connection (rev 01) >> >> >> ff:08.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D QPI Link 0 (rev 01) >> >> >> ff:08.2 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon >> >> >> E3 v4/Xeon D QPI Link 0 (rev 01) >> >> >> ff:08.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D QPI Link 0 (rev 01) >> >> >> ff:09.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D QPI Link 1 (rev 01) >> >> >> ff:09.2 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon >> >> >> E3 v4/Xeon D QPI Link 1 (rev 01) >> >> >> ff:09.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D QPI Link 1 (rev 01) >> >> >> ff:0b.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D R3 QPI Link 0/1 (rev 01) >> >> >> ff:0b.1 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon >> >> >> E3 v4/Xeon D R3 QPI Link 0/1 (rev 01) >> >> >> ff:0b.2 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon >> >> >> E3 v4/Xeon D R3 QPI Link 0/1 (rev 01) >> >> >> ff:0b.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D R3 QPI Link Debug (rev 01) >> >> >> ff:0c.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> ff:0c.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> ff:0c.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> ff:0c.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> ff:0c.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> ff:0c.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> ff:0c.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> ff:0c.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> ff:0d.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> ff:0d.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> ff:0d.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> ff:0d.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> ff:0f.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> ff:0f.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> ff:0f.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> ff:0f.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> ff:0f.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> ff:0f.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> ff:0f.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) >> >> >> ff:10.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D R2PCIe Agent (rev 01) >> >> >> ff:10.1 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon >> >> >> E3 v4/Xeon D R2PCIe Agent (rev 01) >> >> >> ff:10.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Ubox (rev 01) >> >> >> ff:10.6 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon >> >> >> E3 v4/Xeon D Ubox (rev 01) >> >> >> ff:10.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Ubox (rev 01) >> >> >> ff:12.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Home Agent 0 (rev 01) >> >> >> ff:12.1 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon >> >> >> E3 v4/Xeon D Home Agent 0 (rev 01) >> >> >> ff:12.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Home Agent 1 (rev 01) >> >> >> ff:12.5 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon >> >> >> E3 v4/Xeon D Home Agent 1 (rev 01) >> >> >> ff:13.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Target >> >> >> Address/Thermal/RAS (rev 01) >> >> >> ff:13.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Target >> >> >> Address/Thermal/RAS (rev 01) >> >> >> ff:13.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel Target Address >> >> >> Decoder (rev 01) >> >> >> ff:13.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel Target Address >> >> >> Decoder (rev 01) >> >> >> ff:13.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Broadcast (rev 01) >> >> >> ff:13.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Global Broadcast (rev 01) >> >> >> ff:14.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 0 Thermal >> >> >> Control (rev 01) >> >> >> ff:14.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 1 Thermal >> >> >> Control (rev 01) >> >> >> ff:14.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 0 Error (rev >> >> >> 01) >> >> >> ff:14.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 1 Error (rev >> >> >> 01) >> >> >> ff:14.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface (rev 01) >> >> >> ff:14.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface (rev 01) >> >> >> ff:14.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface (rev 01) >> >> >> ff:14.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface (rev 01) >> >> >> ff:16.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Target Address/Thermal/RAS (rev 01) >> >> >> ff:16.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Target Address/Thermal/RAS (rev 01) >> >> >> ff:16.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Channel Target Address Decoder (rev 01) >> >> >> ff:16.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Channel Target Address Decoder (rev 01) >> >> >> ff:16.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Broadcast (rev 01) >> >> >> ff:16.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Global Broadcast (rev 01) >> >> >> ff:17.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 1 - Channel 0 Thermal >> >> >> Control (rev 01) >> >> >> ff:17.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 1 - Channel 1 Thermal >> >> >> Control (rev 01) >> >> >> ff:17.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 1 - Channel 0 Error (rev >> >> >> 01) >> >> >> ff:17.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Memory Controller 1 - Channel 1 Error (rev >> >> >> 01) >> >> >> ff:17.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface (rev 01) >> >> >> ff:17.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface (rev 01) >> >> >> ff:17.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface (rev 01) >> >> >> ff:17.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface (rev 01) >> >> >> ff:1e.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) >> >> >> ff:1e.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) >> >> >> ff:1e.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) >> >> >> ff:1e.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) >> >> >> ff:1e.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) >> >> >> ff:1f.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) >> >> >> ff:1f.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 >> >> >> v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) >> >> >> >> >> >> >> >> >> # ethtool -i S8P0 >> >> >> driver: ixgbe >> >> >> version: 4.4.6 >> >> >> firmware-version: 0x61c10001 >> >> >> expansion-rom-version: >> >> >> bus-info: 0000:82:00.0 >> >> >> supports-statistics: yes >> >> >> supports-test: yes >> >> >> supports-eeprom-access: yes >> >> >> supports-register-dump: yes >> >> >> supports-priv-flags: yes >> >> >> >> >> >> Regards >> >> >> Lukasz >> >> >> >> >> >> >> >> >> > Thanks, >> >> >> > Tal >> >> >> >> >> >> > On Thu, Oct 6, 2016 at 1:06 PM, Łukasz Chrustek >> >> >> > <luk...@chrustek.net> >> >> >> wrote: >> >> >> >> >> >> > Hello, >> >> >> >> >> >> > I have strange problem, my system has two 12-core >> >> >> > procesors, when setting smp_affinity and rps_cpus to spread >> >> >> > load accros all cores, I get following: >> >> >> >> >> >> > ethtool -S S8P0 |grep -E "rx.*pack" >> >> >> > rx_packets: 28059349668 >> >> >> > rx_fcoe_packets: 0 >> >> >> > rx_queue_0_packets: 1739671933 >> >> >> > rx_queue_1_packets: 1740406739 >> >> >> > rx_queue_2_packets: 1945709824 >> >> >> > rx_queue_3_packets: 1741279872 >> >> >> > rx_queue_4_packets: 1739593471 >> >> >> > rx_queue_5_packets: 1740392727 >> >> >> > rx_queue_6_packets: 1740513001 >> >> >> > rx_queue_7_packets: 1740921491 >> >> >> > rx_queue_8_packets: 1741525459 >> >> >> > rx_queue_9_packets: 1739646341 >> >> >> > rx_queue_10_packets: 1741068593 >> >> >> > rx_queue_11_packets: 1741374592 >> >> >> > rx_queue_12_packets: 1739709430 >> >> >> > rx_queue_13_packets: 1747586478 >> >> >> > rx_queue_14_packets: 1740398506 >> >> >> > rx_queue_15_packets: 1739546993 >> >> >> > rx_queue_16_packets: 690 >> >> >> > rx_queue_17_packets: 455 >> >> >> > rx_queue_18_packets: 653 >> >> >> > rx_queue_19_packets: 411 >> >> >> > rx_queue_20_packets: 510 >> >> >> > rx_queue_21_packets: 514 >> >> >> > rx_queue_22_packets: 588 >> >> >> > rx_queue_23_packets: 431 >> >> >> >> >> >> >> >> >> > but for tx: >> >> >> >> >> >> >> >> >> > ethtool -S S8P0 |grep -E "tx.*pack" >> >> >> > tx_packets: 23697264990 >> >> >> > tx_fcoe_packets: 0 >> >> >> > tx_queue_0_packets: 897068903 >> >> >> > tx_queue_1_packets: 897204633 >> >> >> > tx_queue_2_packets: 1705630673 >> >> >> > tx_queue_3_packets: 897124425 >> >> >> > tx_queue_4_packets: 897124753 >> >> >> > tx_queue_5_packets: 897114279 >> >> >> > tx_queue_6_packets: 897379101 >> >> >> > tx_queue_7_packets: 897218157 >> >> >> > tx_queue_8_packets: 904811070 >> >> >> > tx_queue_9_packets: 897124394 >> >> >> > tx_queue_10_packets: 901837638 >> >> >> > tx_queue_11_packets: 897419500 >> >> >> > tx_queue_12_packets: 1241034979 >> >> >> > tx_queue_13_packets: 1247080319 >> >> >> > tx_queue_14_packets: 1247499026 >> >> >> > tx_queue_15_packets: 1240739566 >> >> >> > tx_queue_16_packets: 891720194 >> >> >> > tx_queue_17_packets: 891610281 >> >> >> > tx_queue_18_packets: 891814848 >> >> >> > tx_queue_19_packets: 891770115 >> >> >> > tx_queue_20_packets: 891746387 >> >> >> > tx_queue_21_packets: 891680822 >> >> >> > tx_queue_22_packets: 891775521 >> >> >> > tx_queue_23_packets: 891735428 >> >> >> >> >> >> >> >> >> > Can You provide some clue to spread rx_queue for all cores >> >> >> > not only first 16 ? >> >> >> >> >> >> >> >> -- >> Pozdrawiam, >> Łukasz Chrustek -- Pozdrawiam, Łukasz Chrustek ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
