Re: What does adapter-stats.mpc[] report for ixgbe?
On Mon, May 7, 2012 at 10:16 PM, Jack Vogel jfvo...@gmail.com wrote:
Juli is correct, the FIFO is not partitioned by the driver queues as they
exist in the current driver, its only seperated into the 3 parts I
mentioned.
I am running ixgbe code from 8.1, and IXGBE_FDIR isn't defined, so is
the following doing something to take away some Rx memory:
if (adapter-msix 1) {
/* Enable Enhanced MSIX mode */
gpie |= IXGBE_GPIE_MSIX_MODE;
gpie |= IXGBE_GPIE_EIAME | IXGBE_GPIE_PBA_SUPPORT |
IXGBE_GPIE_OCD;
}
-vijay
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org
Re: What does adapter-stats.mpc[] report for ixgbe?
No, that's just standard setup stuff for MSIX.
Jack
On Tue, May 8, 2012 at 12:08 AM, Vijay Singh vijju.si...@gmail.com wrote:
On Mon, May 7, 2012 at 10:16 PM, Jack Vogel jfvo...@gmail.com wrote:
Juli is correct, the FIFO is not partitioned by the driver queues as they
exist in the current driver, its only seperated into the 3 parts I
mentioned.
I am running ixgbe code from 8.1, and IXGBE_FDIR isn't defined, so is
the following doing something to take away some Rx memory:
if (adapter-msix 1) {
/* Enable Enhanced MSIX mode */
gpie |= IXGBE_GPIE_MSIX_MODE;
gpie |= IXGBE_GPIE_EIAME | IXGBE_GPIE_PBA_SUPPORT |
IXGBE_GPIE_OCD;
}
-vijay
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org
Re: What does adapter-stats.mpc[] report for ixgbe?
This is 'missed packet count', the index has actually been misinterpreted in the code for a while, it was mistakenly associated with queues, but its really per packet buffer, and there are only more than one when there are multiple traffic classes (ala DCB). Even so, only MPC(0) should get incremented anyway. Here is a description: Packets are missed when the receive FIFO has insufficient space to store the incoming packet. This may be caused due to insufficient buffers allocated, or because there is insufficient bandwidth on the IO bus. Events setting this counter also set the receiver overrun interrupt (RXO). These registers do not increment if receive is not enabled and count only packets that would have been posted to the SW driver. Hope this illuminates :) Cheers! Jack On Mon, May 7, 2012 at 5:09 PM, Vijay Singh vijju.si...@gmail.com wrote: Hi, a question for jfv@ or whoever else is familiar with the ixgbe driver - I am looking at a system where the adapter reports a large number of ierrors that I traced to this stat. What does this mean? Intuitively it seems like the receive ring isn't being drained fast enough but wanted to confirm with the experts. -vijay ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org
Re: What does adapter-stats.mpc[] report for ixgbe?
While we're on the subject, I've had some confusion for some time now: On Mon, May 7, 2012 at 5:25 PM, Jack Vogel jfvo...@gmail.com wrote: Packets are missed when the receive FIFO has insufficient space to store the incoming packet. This means the on-card FIFO, i.e. the fixed-size FIFO that is divided into receive and transmit space by the driver, right? And the separate receive no buffers counter records instances where the packet made it to the card but there was no buffer associated with a receive descriptor to write it to? That has been my interpretation of the documentation and the numbers I've seen in practice, but I've seen conflicting interpretations and would much rather hear it from the horse's mouth :) Thanks, Juli. ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org
Re: What does adapter-stats.mpc[] report for ixgbe?
On Mon, May 7, 2012 at 5:31 PM, Juli Mallett jmall...@freebsd.org wrote: While we're on the subject, I've had some confusion for some time now: On Mon, May 7, 2012 at 5:25 PM, Jack Vogel jfvo...@gmail.com wrote: Packets are missed when the receive FIFO has insufficient space to store the incoming packet. This means the on-card FIFO, i.e. the fixed-size FIFO that is divided into receive and transmit space by the driver, right? And the separate receive no buffers counter records instances where the packet made it to the card but there was no buffer associated with a receive descriptor to write it to? That has been my interpretation of the documentation and the numbers I've seen in practice, but I've seen conflicting interpretations and would much rather hear it from the horse's mouth :) Thanks, Juli. Gee, not sure if I like being called a horse :) But yes you are correct. When you have multiple traffic classes, as it is when you have DCB, then the packet buffer may be partitioned into up to 8 parts. This would let you seperate say FCOE and iSCSI from normal net traffic. But for now in FreeBSD its just one which is divided into 3 parts: TX, RX, and FDIR (flow director). Cheers, Jack ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org
Re: What does adapter-stats.mpc[] report for ixgbe?
normal net traffic. But for now in FreeBSD its just one which is divided into 3 parts: TX, RX, and FDIR (flow director). Jack, does the sw driver control in any way the partitioning of the FIFO? I guess enabling 2 hw queues splits the FIFO in half. But otherwise does the driver control this in any way? -vijay ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org
Re: What does adapter-stats.mpc[] report for ixgbe?
On Mon, May 7, 2012 at 9:42 PM, Vijay Singh vijju.si...@gmail.com wrote: normal net traffic. But for now in FreeBSD its just one which is divided into 3 parts: TX, RX, and FDIR (flow director). Jack, does the sw driver control in any way the partitioning of the FIFO? I guess enabling 2 hw queues splits the FIFO in half. But otherwise does the driver control this in any way? I don't believe that multiple queues splits the FIFO (Jack can correct me if I'm wrong.) This is a small (very small) chunk of low-latency memory on the NIC itself that is used to store the packets as they come in off the wire before they are moved to a receive descriptor. The driver does have a way of partitioning the space between transmit and receive, look for PBA in the drivers. In some cases if you're doing mostly-transmit or mostly-receive it can be very helpful to tweak these values, but in the case where you're running out of receive FIFO space constantly, it's (in my limited experience) more likely to be a problem with bandwidth or latency between the NIC and main memory, causing backpressure within the NIC as it tries to move packets to receive buffers (which are the ones allocated in main, i.e. host, memory.) ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org
Re: What does adapter-stats.mpc[] report for ixgbe?
Juli is correct, the FIFO is not partitioned by the driver queues as they exist in the current driver, its only seperated into the 3 parts I mentioned. Jack On Mon, May 7, 2012 at 9:55 PM, Juli Mallett jmall...@freebsd.org wrote: On Mon, May 7, 2012 at 9:42 PM, Vijay Singh vijju.si...@gmail.com wrote: normal net traffic. But for now in FreeBSD its just one which is divided into 3 parts: TX, RX, and FDIR (flow director). Jack, does the sw driver control in any way the partitioning of the FIFO? I guess enabling 2 hw queues splits the FIFO in half. But otherwise does the driver control this in any way? I don't believe that multiple queues splits the FIFO (Jack can correct me if I'm wrong.) This is a small (very small) chunk of low-latency memory on the NIC itself that is used to store the packets as they come in off the wire before they are moved to a receive descriptor. The driver does have a way of partitioning the space between transmit and receive, look for PBA in the drivers. In some cases if you're doing mostly-transmit or mostly-receive it can be very helpful to tweak these values, but in the case where you're running out of receive FIFO space constantly, it's (in my limited experience) more likely to be a problem with bandwidth or latency between the NIC and main memory, causing backpressure within the NIC as it tries to move packets to receive buffers (which are the ones allocated in main, i.e. host, memory.) ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org
