On Sun, 2017-10-22 at 20:14 +0100, Ard Biesheuvel wrote: > Hello all, > > I am working on upstreaming a network driver for a Socionext SoC, and > I am having some trouble figuring out why my TX performance is > horrible when booting a Debian Stretch rootfs, while booting a Ubuntu > 17.04 rootfs works absolutely fine. Note that this is using the exact > same kernel image, booted off the network. > > Under Ubuntu, I get the following iperf results from the box to my AMD > Seattle based devbox with a 1 Gbit switch in between. (The NIC in > question is also 1 Gbit) > > > $ sudo iperf -c dogfood.local -r > ------------------------------------------------------------ > Server listening on TCP port 5001 > TCP window size: 85.3 KByte (default) > ------------------------------------------------------------ > ------------------------------------------------------------ > Client connecting to dogfood.local, TCP port 5001 > TCP window size: 748 KByte (default) > ------------------------------------------------------------ > [ 5] local 192.168.1.112 port 51666 connected with 192.168.1.106 port 5001 > [ ID] Interval Transfer Bandwidth > [ 5] 0.0-10.0 sec 1.07 GBytes 920 Mbits/sec > [ 4] local 192.168.1.112 port 5001 connected with 192.168.1.106 port 33048 > [ 4] 0.0-10.0 sec 1.10 GBytes 940 Mbits/sec > > Booting the *exact* same kernel into a Debian based rootfs results in > the following numbers > $ sudo iperf -c dogfood.local -r > ------------------------------------------------------------ > Server listening on TCP port 5001 > TCP window size: 85.3 KByte (default) > ------------------------------------------------------------ > ------------------------------------------------------------ > Client connecting to dogfood.local, TCP port 5001 > TCP window size: 85.0 KByte (default) > ------------------------------------------------------------ > [ 5] local 192.168.1.112 port 40132 connected with 192.168.1.106 port 5001 > [ ID] Interval Transfer Bandwidth > [ 5] 0.0-10.1 sec 4.12 MBytes 3.43 Mbits/sec > [ 4] local 192.168.1.112 port 5001 connected with 192.168.1.106 port 33068 > [ 4] 0.0-10.0 sec 1.10 GBytes 939 Mbits/sec > > The ifconfig stats look perfectly fine to me (TX errors 0 dropped 0 > overruns 0 carrier 0 collisions 0). During the TX test, the CPUs are > almost completely idle. (This system has 24 cores, but not > particularly powerful ones.) > > This test is based on v4.14-rc4, but v4.13 gives the same results. > > Could anyone please shed a light on this? What tuning parameters > and/or stats should I be looking at? I am a seasoned kernel developer > but a newbie when it comes to networking, so hopefully I just need a > nudge to go looking in the right place.
This description smells a problem with TX completions being deferred. TX interrupts being lost or deferred too much.
