Module Name: src
Committed By: msaitoh
Date: Tue Feb 20 07:24:37 UTC 2018
Modified Files:
src/sys/dev/pci/ixgbe: ixgbe.c ixgbe_type.h
Log Message:
- Fix a bug that RX may stall on heavy load on ixg(4) derived from FreeBSD's
AIM (Auto Interrupt Moderation) bug.
When I use a machine as a NFS client, sometimes one of queue pairs doesn't
get any interrupt other than every second tick via ixgbe_local_timer1().
When the problem occured, the queue pair's hw.ixgM.qN.interrupt_rate is
always 500000. When this problem occuring, set hw.ixgM.qN.interrupt_rate lower
than 166667 recover from stall. i.e.:
sysctl -w hw.ixgM.qN.interrupt_rate=166667 (don't revocer)
sysctl -w hw.ixgM.qN.interrupt_rate=166666 (recover)
Relatios between the interrupt_rate and EICR's ITR_INTERVAL field is as
follows:
int_rate | EICR[11:0] | interval in us | recover |
|(ITR_INTERVAL)| (10G and 1G) | |
---------+--------------+----------------+---------+
500000 | 0x008(0) | 2 | not |
166667 | 0x010(1) | 4 | not |
166666 | 0x018(2) | 6 | recover |
The reason why int_rate becomes 500000 is that xgbe_tx_eof() doesn't
increment rxr->packets(*1). Even if we fix rxr->packets' bug, interrupt_rate
might become greater than 166666 and it might cause stall.
While reading datasheets, knakahara noticed a section titled with "ITR
Affect on RSC Functionality". It says "When RSC is enabled on specific RX
queues, the associated ITR interval with these queus must be enabled and must
be larger (in time uints) than RSC delay". Currently, RSC_DELAY field in the
GPIE register is 0 and it means 4us for 10G and 1G. The greater ITR_INTERVAL
value of 4us is 6us == 166666. Yes, BINGO!
This description is noted in 82599 and newer datasheets and not in 82598
datasheet. I don't know if 82598 has this limitation but, I apply this
limitation all of chips.
(*1) Note that this bug is going to be fixed in the next commit to distinct
between two different bugs.
- The bitfield of EITR register is different between 82598 and others.
Only ixgbe_msix_que() taken care of it. Make new function ixgbe_eitr_write()
and use it in all of functions which modify ITR_INTERVAL.
XXX pullup-8
To generate a diff of this commit:
cvs rdiff -u -r1.123 -r1.124 src/sys/dev/pci/ixgbe/ixgbe.c
cvs rdiff -u -r1.30 -r1.31 src/sys/dev/pci/ixgbe/ixgbe_type.h
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
