>-----Original Message----- >From: Андрей Василишин [mailto:[email protected]] >Sent: Tuesday, June 10, 2014 8:35 AM >To: Tantilov, Emil S; [email protected] >Subject: Re: [E1000-devel] 82599EB big packet loss > > >Emil, thaks for reply! >>> with one of this modules (don't remember which one): >Intel SFP+ SR FTLX8571D3BCV-IT or Intel SFP+ SR AFBR- >703SDZ-IN2 >> If the issue is only seen with specific module, then it >would help to know the exact model. >> >I have three servers with 82599 NICs (two with 82599EB and >one with 82599ES) in one data-centre, all with the same behavior. > >>> When OUTbount traffic up to ~9800 Mbit/s INbound traffic >appear packet loss up to 30% >> >> From the stats, some of the counters that maybe related >to the packet loss: >> >> >>> rx_no_dma_resources: 68 >> >> Because this counter increments, but xoff counters are 0, >I'm guessing you don't have flow control enabled. This >basically means that the driver has no free Rx descriptors >in the receive queue. >> > >Just enable ethtool -A eth0 rx on tx on > >>> fdir_overflow: 1424 >> >> You have many sessions which exceed the space allocated >for Fdir filters. You can try and increase it using the FdirPballoc parameter. > > >and increase FdirPballoc > >Now I have such options: >options ixgbe RSS=8 DCA=2 LLIPort=80 allow_unsupported_sfp=1 >FdirPballoc=3 > >The problem remained :(
The counters you provided are from a smaller sample size (judging by the total packets), but they look very similar - at least with respect to fdir_overflow. If the number of sessions exceeds the buffer allowed by the driver for storing the Fdir filters, you may be better off creating your own using ethtool -U Also some other suggestions you may want to try: 1. If not already, try spreading the interrupts for each CPU using the set_irq_affinity script included with the driver instead of irqbalance. 2. Because flow control is not negotiated, you may want to check if your link partner supports it as well (usually you can tell if you check dmesg for the "Link Up" event - it will include information about whether flow control is enabled and in what direction). 3. LLI - I'm not sure why you enable it, but this could be adding to your trouble on Rx. You may want to experiment with ethtool -C rx-usec and allow for more ints/sec although this will affect your entire traffic, not just port 80. 4. Try and disable LRO (you can tell by the hw_rsc counters) - ethtool -K ethX lro off. In addition increasing the number of Rx descriptors may help (ethtool -G) 5. Last but not least because you are using unsupported module - it may be a good idea to try with SFP+ module that loads without the need for allow_unsupported_sfp. This will help you rule out incompatibilities with the SFP+ module. Performance is a tricky business and finding the parameters that work best for your environment may take some time. What I would suggest is start with a clean ixgbe driver and then use module parameters and/or ethtool to adjust settings one by one to see how your performance is affected by each change. Thanks, Emil ------------------------------------------------------------------------------ HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions Find What Matters Most in Your Big Data with HPCC Systems Open Source. Fast. Scalable. Simple. Ideal for Dirty Data. Leverages Graph Analysis for Fast Processing & Easy Data Exploration http://p.sf.net/sfu/hpccsystems _______________________________________________ E1000-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
