On Sat, Mar 16, 2024 at 12:38 PM Ivan Vecera <[email protected]> wrote: > As for ice bug fixed by commit b7306b42beaf ("ice: manage interrupts > during poll exit") followed by commit 23be7075b318 ("ice: fix software > generating extra interrupts") I'm seeing the similar issue also with > i40e driver. > > In certain situation when busy-loop is enabled together with adaptive > coalescing, the driver occasionally misses that there are outstanding > descriptors to clean when exiting busy poll. > > Try to catch the remaining work by triggering a software interrupt > when exiting busy poll. No extra interrupts will be generated when > busy polling is not used. > > The issue was found when running sockperf ping-pong tcp test with > adaptive coalescing and busy poll enabled (50 as value busy_pool > and busy_read sysctl knobs) and results in huge latency spikes > with more than 100000us. > > The fix is inspired from the ice driver and do the following: > 1) During napi poll exit in case of busy-poll (napo_complete_done() > returns false) this is recorded to q_vector that we were in busy > loop. > 2) Extends i40e_buildreg_itr() to be able to add an enforced software > interrupt into built value > 2) In i40e_update_enable_itr() enforces a software interrupt trigger > if we are exiting busy poll to catch any pending clean-ups > 3) Reuses unused 3rd ITR (interrupt throttle) index and set it to > 20K interrupts per second to limit the number of these sw interrupts. > > Test results > ============ > Prior: > [root@dell-per640-07 net]# sockperf ping-pong -i 10.9.9.1 --tcp -m 1000 > --mps=max -t 120 > sockperf: == version #3.10-no.git == > sockperf[CLIENT] send on:sockperf: using recvfrom() to block on socket(s) > > [ 0] IP = 10.9.9.1 PORT = 11111 # TCP > sockperf: Warmup stage (sending a few dummy messages)... > sockperf: Starting test... > sockperf: Test end (interrupted by timer) > sockperf: Test ended > sockperf: [Total Run] RunTime=119.999 sec; Warm up time=400 msec; > SentMessages=2438563; ReceivedMessages=2438562 > sockperf: ========= Printing statistics for Server No: 0 > sockperf: [Valid Duration] RunTime=119.549 sec; SentMessages=2429473; > ReceivedMessages=2429473 > sockperf: ====> avg-latency=24.571 (std-dev=93.297, mean-ad=4.904, > median-ad=1.510, siqr=1.063, cv=3.797, std-error=0.060, 99.0% ci=[24.417, > 24.725]) > sockperf: # dropped messages = 0; # duplicated messages = 0; # out-of-order > messages = 0 > sockperf: Summary: Latency is 24.571 usec > sockperf: Total 2429473 observations; each percentile contains 24294.73 > observations > sockperf: ---> <MAX> observation = 103294.331 > sockperf: ---> percentile 99.999 = 45.633 > sockperf: ---> percentile 99.990 = 37.013 > sockperf: ---> percentile 99.900 = 35.910 > sockperf: ---> percentile 99.000 = 33.390 > sockperf: ---> percentile 90.000 = 28.626 > sockperf: ---> percentile 75.000 = 27.741 > sockperf: ---> percentile 50.000 = 26.743 > sockperf: ---> percentile 25.000 = 25.614 > sockperf: ---> <MIN> observation = 12.220 > > After: > [root@dell-per640-07 net]# sockperf ping-pong -i 10.9.9.1 --tcp -m 1000 > --mps=max -t 120 > sockperf: == version #3.10-no.git == > sockperf[CLIENT] send on:sockperf: using recvfrom() to block on socket(s) > > [ 0] IP = 10.9.9.1 PORT = 11111 # TCP > sockperf: Warmup stage (sending a few dummy messages)... > sockperf: Starting test... > sockperf: Test end (interrupted by timer) > sockperf: Test ended > sockperf: [Total Run] RunTime=119.999 sec; Warm up time=400 msec; > SentMessages=2400055; ReceivedMessages=2400054 > sockperf: ========= Printing statistics for Server No: 0 > sockperf: [Valid Duration] RunTime=119.549 sec; SentMessages=2391186; > ReceivedMessages=2391186 > sockperf: ====> avg-latency=24.965 (std-dev=5.934, mean-ad=4.642, > median-ad=1.485, siqr=1.067, cv=0.238, std-error=0.004, 99.0% ci=[24.955, > 24.975]) > sockperf: # dropped messages = 0; # duplicated messages = 0; # out-of-order > messages = 0 > sockperf: Summary: Latency is 24.965 usec > sockperf: Total 2391186 observations; each percentile contains 23911.86 > observations > sockperf: ---> <MAX> observation = 195.841 > sockperf: ---> percentile 99.999 = 45.026 > sockperf: ---> percentile 99.990 = 39.009 > sockperf: ---> percentile 99.900 = 35.922 > sockperf: ---> percentile 99.000 = 33.482 > sockperf: ---> percentile 90.000 = 28.902 > sockperf: ---> percentile 75.000 = 27.821 > sockperf: ---> percentile 50.000 = 26.860 > sockperf: ---> percentile 25.000 = 25.685 > sockperf: ---> <MIN> observation = 12.277 > > Fixes: 0bcd952feec7 ("ethernet/intel: consolidate NAPI and NAPI exit") > Reported-by: Hugo Ferreira <[email protected]> > Reviewed-by: Michal Schmidt <[email protected]>
Yes, I am OK with v4. Michal > Signed-off-by: Ivan Vecera <[email protected]>
