[E1000-devel] Fragmented UDP packets trigger rx_missed_errors on 82599EB
Hi While investigating a upper level network issue, I found out the root cause may be triggered by packet loss at NIC level, showed by rx_missed_errors. kernel: linux-2.6.32-358.el6.x86_64 server: iperf -s -B 192.168.5.1 -u client: iperf -c 192.168.5.1 -u -b 10G -i 1 -t 1000 -P 12 -l 3k Use -l to specify buffers large than MTU to create fragmented IP packets. 1. Tune rx ring from 512 to max 4096 does help for single flow, but still got great rx_missed_errors from multiple flows. 2. Using latest net-next 4.0.0-rc4 shows the same effect. 3. Got 9.4Gbits/sec even though rx_missed_errors shows NIC level packets drop. rx_missed_errors value comes from RXMPC, where 82599 data sheet 8.2.3.5.1 says: "Missed packet interrupt is activated for each received packet that overflows the Rx packet buffer (overrun). he packet is dropped and also increments the associated RXMPC[n] counter." I'm not sure it means my env is mis-configured or anything I'm missing obviously. Any hints? Attached several logs as below. # ethtool -S eth4 NIC statistics: rx_packets: 1047869017 tx_packets: 206275776 rx_bytes: 110268576 tx_bytes: 289198212456 rx_pkts_nic: 1047200292 tx_pkts_nic: 206275773 rx_bytes_nic: 1907927064202 tx_bytes_nic: 290023317512 lsc_int: 17 tx_busy: 0 non_eop_descs: 0 rx_errors: 0 tx_errors: 0 rx_dropped: 0 tx_dropped: 0 multicast: 0 broadcast: 4310 rx_no_buffer_count: 0 collisions: 0 rx_over_errors: 0 rx_crc_errors: 0 rx_frame_errors: 0 hw_rsc_aggregated: 0 hw_rsc_flushed: 0 fdir_match: 0 fdir_miss: 6545204 fdir_overflow: 0 rx_fifo_errors: 0 rx_missed_errors: 638609576 < tx_aborted_errors: 0 tx_carrier_errors: 0 tx_fifo_errors: 0 tx_heartbeat_errors: 0 tx_timeout_count: 0 tx_restart_queue: 0 rx_long_length_errors: 0 rx_short_length_errors: 0 tx_flow_control_xon: 174182 rx_flow_control_xon: 0 tx_flow_control_xoff: 946044 # numactl --hardware available: 4 nodes (0-3) node 0 cpus: 0 1 2 3 4 20 21 22 23 24 node 0 size: 24466 MB node 0 free: 22444 MB node 1 cpus: 5 6 7 8 9 25 26 27 28 29 node 1 size: 16384 MB node 1 free: 15831 MB node 2 cpus: 10 11 12 13 14 30 31 32 33 34 node 2 size: 16384 MB node 2 free: 15791 MB node 3 cpus: 15 16 17 18 19 35 36 37 38 39 node 3 size: 24576 MB node 3 free: 22508 MB node distances: node 0 1 2 3 0: 10 21 31 31 1: 21 10 31 31 2: 31 31 10 21 3: 31 31 21 10 # ethtool -g eth4 Ring parameters for eth4: Pre-set maximums: RX: 4096 RX Mini: 0 RX Jumbo: 0 TX: 4096 Current hardware settings: RX: 4096 < I tweak it from 512 to max 4096, it helps for single flow, but still not good for multiple flows. RX Mini: 0 RX Jumbo: 0 TX: 512 # ethtool -a eth4 Pause parameters for eth4: Autonegotiate: on RX: on TX: on # ethtool -c eth4 Coalesce parameters for eth4: Adaptive RX: off TX: off stats-block-usecs: 0 sample-interval: 0 pkt-rate-low: 0 pkt-rate-high: 0 rx-usecs: 1 rx-frames: 0 rx-usecs-irq: 0 rx-frames-irq: 0 tx-usecs: 0 tx-frames: 0 tx-usecs-irq: 0 tx-frames-irq: 0 rx-usecs-low: 0 rx-frame-low: 0 tx-usecs-low: 0 tx-frame-low: 0 rx-usecs-high: 0 rx-frame-high: 0 tx-usecs-high: 0 tx-frame-high: 0 # lspci -vv (Assuming I'm using 84:00.0) 84:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01) Subsystem: Intel Corporation Ethernet Server Adapter X520-2 Physical Slot: 803 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- TAbort- SERR- http://goparallel.sourceforge.net/ ___ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
Re: [E1000-devel] [net-next 8/9] i40e/i40evf: Bump i40e/i40evf version
On Wed, Apr 01, 2015 at 09:52:35 -0700, Jesse Brandeburg wrote: > I think the piece you're missing is that you need to run the QCU (QSFP+ > Configuration Utility) to switch the port from 40G to 4x10 mode. Thanks. I now have ens5f[0-3]. But also: [ 329.351781] i40e :81:00.0 ens5f0: the driver failed to link because an unqualified module was detected. on all ports. I guess the breakout cable is in the wrong port (we have the 2-port version with an Intel breakout and a 3rd party QSFP). I'll put the breakout cable in the other port tomorrow. rvdp -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
Re: [E1000-devel] [net-next 8/9] i40e/i40evf: Bump i40e/i40evf version
On Wed, 1 Apr 2015 11:52:37 +0200 Ronald van der Pol wrote: > On Thu, Mar 26, 2015 at 23:01:46 -0700, Jeff Kirsher wrote: > > > netdev is not the right mailing list for this question. > > Adding e1000-devel mailing list... > > Sorry about that. I still have problems with getting the Intel X4DACBL3 > QSFP to 4x SFP breakout cable working with the i40e. I have also upgraded > the NVM, but it did not help. Below is the modprobe output. I think the piece you're missing is that you need to run the QCU (QSFP+ Configuration Utility) to switch the port from 40G to 4x10 mode. At that point the interface will show up as four physical functions 81:0.0 - 81:0.3 > > [root@boron src]# ethtool -i ens5f0 > driver: i40e > version: 1.2.37 > firmware-version: f4.33.31377 a1.2 n4.42 e1932 This is the right NVM to run QCU on top of. > Apr 1 12:21:27 boron kernel: i40e: Intel(R) Ethernet Connection XL710 > Network Driver - version 1.2.37 This is a good driver to be running. :-) > PS I have a 3rd party QSFP-QSFP DAC cable + Intel X4DACBL3 inserted. Might > the 3rd party DAC cable confuse the driver? I need to travel to get > physical access to the server, so I cannot easily pull the cable. I don't know, try QCU first. The adapter and driver can't automatically switch to 4x10 mode. > PPS I understand: > - 3rd party optics are not supported > - max of 4 mac addresses, so 4x10 is OK, 4x10 + 1x40 is not OK > Is this correct? right, AFAIK -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
[E1000-devel] Produtos com até 60% OFF! No Dia da Mentira o desconto é de verdade!
Acesse: http://promo5fun.com/newapp/link.php?M=903726&N=289&L=221&F=T [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2163&F=T] Veja online [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2050&F=T con...@email.consul.com.br [mailto:con...@email.consul.com.br] aos seus contatos e receba nossos e-mails na Caixa de EntradaDescadastre-se [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2164&F=T [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2165&F=T] | Frigobares [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2166&F=T] | Fogões [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2167&F=T] | Micro-ondas [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2168&F=T] | Climatizadores [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2169&F=T] | Ar Condicionados [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2170&F=T] | Produtos de Embutir [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2171&F=T] | Freezer [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2172&F=T] [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2173&F=T] [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2174&F=T] [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2175&F=T] [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2176&F=T] [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2177&F=T] [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2178&F=T] [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2179&F=T] [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2180&F=T] [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2181&F=T] [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2182&F=T] [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2183&F=T] Ofertas válidas até 01/04/2015, sujeitas a alteração de preço e condições de pagamento. No momento de sua compra informe o seu CEP para calcular o valor do frete, prazo de entrega e disponibilidade de produto e atendimento de nossas transportadoras para a sua região. Na compra de mais de um produto, eles poderão ser entregues separadamente Televendas: (11) 3003-7872. Custo ligação local SAC: 3003 - 0099/ 0800 9700 999 BUD COMERCIO DE ELETRODOMESTICOS LTDA, CNPJ 62.058.318/0007-76 Endereço Sede Rua Olympia Semeraro, 675, Jardim Santa Emilia, São Paulo/ SP à proibido condicionar desconto no preço do bem à aquisição do seguro. A contratação de seguro é opcional. O segurado poderá desistir do seguro contratado no prazo de 7 (sete) dias corridos a contar da emissão do bilhete, ou do efetivo pagamento do prêmio, o que ocorrer por último. Processo SUSEP Garantia Estendida Original: 15414.900511/2014-42 O registro deste plano na SUSEP não implica, por parte da Autarquia, incentivo ou recomendação a sua comercialização. A comercialização de seguro é fiscalizada pela SUSEP, www.susep.gov.br [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2026&F=T] e pelo telefone gratuito SUSEP 0800 021 8484. A relação dos preços dos seguros ofertados pela AIG está disponÃvel, consulte-o. O produto possui restrições e exclusões, por favor consulte as condições gerais para mais informações. A Ãntegra das Condições Gerais está disponÃvel no site www.aig.com.br [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2027&F=T]. O segurado poderá consultar a situação cadastral de seu corretor de seguros no site www.susep.gov.br [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=2026&F=T], por meio do número de seu registro na SUSEP, nome completo, CNPJ ou CPF. Se você não deseja mais receber nossos informativos, Clique aqui para deixar de receber estes e-mails [http://promo5fun.com/newapp/link.php?M=903726&N=289&L=28&F=T] Clique neste link para ser removido: http://promo5fun.com/newapp/unsubscribe.php?M=903726&C=6677a34cff14e697240e32adc84786ac&L=6&N=289 -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
Re: [E1000-devel] [net-next 8/9] i40e/i40evf: Bump i40e/i40evf version
On Thu, Mar 26, 2015 at 23:01:46 -0700, Jeff Kirsher wrote: > netdev is not the right mailing list for this question. > Adding e1000-devel mailing list... Sorry about that. I still have problems with getting the Intel X4DACBL3 QSFP to 4x SFP breakout cable working with the i40e. I have also upgraded the NVM, but it did not help. Below is the modprobe output. [root@boron src]# ethtool -i ens5f0 driver: i40e version: 1.2.37 firmware-version: f4.33.31377 a1.2 n4.42 e1932 bus-info: :81:00.0 supports-statistics: yes supports-test: yes supports-eeprom-access: yes supports-register-dump: yes supports-priv-flags: yes Apr 1 12:21:27 boron kernel: i40e: Intel(R) Ethernet Connection XL710 Network Driver - version 1.2.37 Apr 1 12:21:27 boron kernel: i40e: Copyright (c) 2013 - 2015 Intel Corporation. Apr 1 12:21:27 boron kernel: i40e :81:00.0: f4.33.31377 a1.2 n4.42 e1932 Apr 1 12:21:27 boron kernel: i40e :81:00.0: FCoE capability is disabled Apr 1 12:21:27 boron kernel: i40e :81:00.0: MAC address: 68:05:ca:2e:70:e0 Apr 1 12:21:27 boron kernel: i40e :81:00.0: SAN MAC: 68:05:ca:2e:70:e4 Apr 1 12:21:27 boron kernel: i40e :81:00.0: fcoe queues = 0 Apr 1 12:21:27 boron kernel: i40e :81:00.0: enabling bridge mode: VEB Apr 1 12:21:27 boron kernel: i40e :81:00.0: PHC enabled Apr 1 12:21:27 boron kernel: i40e :81:00.0: PCI-Express: Speed 8.0GT/s Width x8 Apr 1 12:21:27 boron kernel: i40e :81:00.0: get phy abilities failed, aq_err -7, advertised speed settings may not be correct Apr 1 12:21:27 boron kernel: i40e :81:00.0: Features: PF-id[0] VFs: 64 VSIs: 66 QP: 32 RX: 1BUF RSS FD_ATR FD_SB NTUPLE DCB PTP Apr 1 12:21:27 boron kernel: i40e :81:00.1: f4.33.31377 a1.2 n4.42 e1932 Apr 1 12:21:27 boron kernel: i40e :81:00.1: FCoE capability is disabled Apr 1 12:21:27 boron kernel: i40e :81:00.1: MAC address: 68:05:ca:2e:70:e1 Apr 1 12:21:27 boron kernel: i40e :81:00.1: SAN MAC: 68:05:ca:2e:70:e5 Apr 1 12:21:27 boron kernel: i40e :81:00.1: fcoe queues = 0 Apr 1 12:21:27 boron kernel: i40e :81:00.1: enabling bridge mode: VEB Apr 1 12:21:27 boron kernel: i40e :81:00.1: PHC enabled Apr 1 12:21:27 boron kernel: i40e :81:00.1: PCI-Express: Speed 8.0GT/s Width x8 Apr 1 12:21:27 boron kernel: i40e :81:00.1: get phy abilities failed, aq_err -7, advertised speed settings may not be correct Apr 1 12:21:27 boron kernel: i40e :81:00.1: Features: PF-id[1] VFs: 64 VSIs: 66 QP: 32 RX: 1BUF RSS FD_ATR FD_SB NTUPLE DCB PTP PS I have a 3rd party QSFP-QSFP DAC cable + Intel X4DACBL3 inserted. Might the 3rd party DAC cable confuse the driver? I need to travel to get physical access to the server, so I cannot easily pull the cable. PPS I understand: - 3rd party optics are not supported - max of 4 mac addresses, so 4x10 is OK, 4x10 + 1x40 is not OK Is this correct? -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
Re: [E1000-devel] [patch] igb: cleanup igb_enable_mas() a bit
On Tue, Mar 31, 2015 at 08:10:31PM +, Fujinaka, Todd wrote: > I have a fix for this already, if Dan doesn't mind. > I don't mind. regards, dan carpenter -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
