Here's what I did ethtool -A p1p1 rx off tx off ethtool -A p3p1 rx off tx off
Both ethtool -a <interface> and Arista that's pumping data show that RX/TX pause are disabled. I have two cards, each connected to a separate NUMA node, threads pinned, etc. One non-standard thing is that I use single queue only, because any form of multiqueue leads to packet reordering and confuses IDS. An issue that's been hidden for a while in the NSM community. driver (from sourceforge) was loaded with MQ=0 DCA=2 RSS=1 VMDQ=0 InterruptThrottleRate=956 FCoE=0 LRO=0 vxvlan_rx=0 (each option's value given enogh times so it applies to all cards in this system). I could see the same issue sending traffic to just one card. Of course a single core is swamped with ACK-ing hardware IRQ and then doing softIRQ (which seems to be mostly memcpy?). But then again, I don't see errors about lacking buffers (I run with 4096 descriptors). On Fri, Sep 23, 2016 at 9:22 PM, Alexander Duyck <alexander.du...@gmail.com> wrote: > When you say you disabled flow control did you disable it on the > interface that is dropping packets or the other end? You might try > explicitly disabling it on the interface that is dropping packets, > that in turn should enable per-queue drop instead of putting > back-pressure onto the Rx FIFO. > > With flow control disabled on the local port you should see > rx_no_dma_resources start incrementing if the issue is that one of the > Rx rings is not keeping up. > > - Alex > > On Fri, Sep 23, 2016 at 11:09 AM, Michał Purzyński > <michalpurzyns...@gmail.com> wrote: > > xoff was increasing so I disabled flow control. > > > > That's a HP DL360 Gen9 and lspci -vvv tells me cards are connected to x8 > > link, speed is 5GT/s and ASPM is disabled. > > > > Other error counters are still zero. When I compared rx_packets and > > rx_missed_errors it looks like a 38% (!!) packets are getting lost. > > > > Unfortunately HP documentation is a scam and they actively avoid > publishing > > motherboard layout. > > > > Any other place I could look for hints? > > > > > > On Fri, Sep 23, 2016 at 7:01 PM, Alexander Duyck < > alexander.du...@gmail.com> > > wrote: > >> > >> On Fri, Sep 23, 2016 at 1:10 AM, Michał Purzyński > >> <michalpurzyns...@gmail.com> wrote: > >> > Hello. > >> > > >> > On my IDS workload with af_packet I can see rx_missed_errors growing > >> > while > >> > rx_no_buffer_count does not. Basically every other kind of rx_ error > >> > counter is 0, including rx_no_dma_resources. It's an 82599 based card. > >> > > >> > I don't know what to think about that. I went through ixgbe source > code > >> > and > >> > the 82599 datasheet and seems like rx_missed_error means a new packet > >> > overwrote something already in the packet buffer (FIFO queue on the > >> > card) > >> > because there was no more space in it. > >> > > >> > Now, that would happen if there is no place to DMA packets into - but > >> > that > >> > counter does not grow. > >> > > >> > Could you point me to where should I be looking for a problem? > >> > > >> > -- > >> > Michal Purzynski > >> > >> The Rx missed count will increment if you are not able to receive a > >> packet because the Rx FIFO is full. If you are not seeing any > >> rx_no_dma_resources problems it might indicate that the problem is not > >> with providing the DMA resources, but a problem on the bus itself. > >> You might want to double check the slot the device is connected to in > >> order to guarantee that there is a x8 link that supports 5GT/s all the > >> way through to the root complex. > >> > >> - Alex > > > > >
------------------------------------------------------------------------------
_______________________________________________ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
