On Sun, 22 Feb 2026 23:08:43 +0100,
Alexander Bluhm <[email protected]> wrote:
>
> On Sun, Feb 22, 2026 at 10:20:54PM +0100, Kirill A. Korinsky wrote:
> > On Sun, 22 Feb 2026 21:49:53 +0100,
> > Klemens Nanni <[email protected]> wrote:
> > >
> > >
> > > Thanks for testing, I certainly did not expect such a massive drop.
> > >
> > > I'll test here as well and try to see what this is about before
> > > committing anything.
> > >
> > > I wonder, if this is "just" a weak CPU or something else, e.g. specific to
> > > octeon, NIC drivers and/or in your setup.
> > >
> > > Can you share your iperf commands and a dmesg for completeness, please?
> > >
> > > Cc'ing Janne: You tried a similar diff on your boxes; care to share
> > > which ones and whether you saw any noticable change in performance?
> > >
> >
> > I run as simple commands as:
> >
> > iperf3-darwin -t 30 -c 172.31.3.1
>
> This uses only one TCP stream which uses only one CPU. With multiple
> softnet threads there is only locking overhead but no parallelism.
>
> Can you try to run iperf3 with -P5 or -P10 ?
>
> -P, --parallel n
> number of parallel client streams to run. iperf3 will spawn off a
> separate thread for each test stream. Using multiple streams may
> result in higher throughput than a single stream.
>
Sure I can.
without patch:
% iperf3-darwin -t 30 -c 172.31.2.93
...
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.00 sec 1.64 GBytes 468 Mbits/sec 137560 sender
[ 5] 0.00-30.01 sec 1.63 GBytes 468 Mbits/sec receiver
% iperf3-darwin -t 30 -c 172.31.2.93 -P 5
...
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.00 sec 182 MBytes 50.8 Mbits/sec 98464 sender
[ 5] 0.00-30.04 sec 181 MBytes 50.5 Mbits/sec receiver
[ 7] 0.00-30.00 sec 346 MBytes 96.8 Mbits/sec 88328 sender
[ 7] 0.00-30.04 sec 345 MBytes 96.3 Mbits/sec receiver
[ 9] 0.00-30.00 sec 321 MBytes 89.6 Mbits/sec 85432 sender
[ 9] 0.00-30.04 sec 320 MBytes 89.3 Mbits/sec receiver
[ 11] 0.00-30.00 sec 397 MBytes 111 Mbits/sec 904416 sender
[ 11] 0.00-30.04 sec 397 MBytes 111 Mbits/sec receiver
[ 13] 0.00-30.00 sec 421 MBytes 118 Mbits/sec 2259272 sender
[ 13] 0.00-30.04 sec 420 MBytes 117 Mbits/sec receiver
[SUM] 0.00-30.00 sec 1.63 GBytes 466 Mbits/sec 3435912 sender
[SUM] 0.00-30.04 sec 1.62 GBytes 464 Mbits/sec receiver
% iperf3-darwin -t 30 -c 172.31.3.1
...
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.00 sec 1.29 GBytes 370 Mbits/sec 5829 sender
[ 5] 0.00-30.01 sec 1.29 GBytes 369 Mbits/sec receiver
% iperf3-darwin -t 30 -c 172.31.3.1 -P 5
...
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.00 sec 181 MBytes 50.5 Mbits/sec 1485 sender
[ 5] 0.00-30.04 sec 180 MBytes 50.4 Mbits/sec receiver
[ 7] 0.00-30.00 sec 194 MBytes 54.1 Mbits/sec 8677 sender
[ 7] 0.00-30.04 sec 193 MBytes 54.0 Mbits/sec receiver
[ 9] 0.00-30.00 sec 181 MBytes 50.5 Mbits/sec 4365 sender
[ 9] 0.00-30.04 sec 180 MBytes 50.4 Mbits/sec receiver
[ 11] 0.00-30.00 sec 614 MBytes 172 Mbits/sec 1485 sender
[ 11] 0.00-30.04 sec 613 MBytes 171 Mbits/sec receiver
[ 13] 0.00-30.00 sec 177 MBytes 49.4 Mbits/sec 1485 sender
[ 13] 0.00-30.04 sec 176 MBytes 49.3 Mbits/sec receiver
[SUM] 0.00-30.00 sec 1.31 GBytes 376 Mbits/sec 17497 sender
[SUM] 0.00-30.04 sec 1.31 GBytes 375 Mbits/sec receiver
and make config when obj and src are both in NFS:
gw$ cd /usr/src/sys/arch/$(machine)/compile/GENERIC.MP
gw$ doas time make config
cd /usr/obj/sys/arch/octeon/compile/GENERIC.MP && config -s /usr/src/sys -b
/usr/src/sys/arch/octeon/compile/GENERIC.MP/obj
/usr/src/sys/arch/octeon/conf/GENERIC.MP
5.33 real 1.48 user 0.68 sys
gw$
ping from my laptop:
--- 172.31.2.1 ping statistics ---
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 1.347/1.574/1.814/0.146 ms
and ping from machine which I've used to run iperf:
--- 172.31.3.1 ping statistics ---
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.700/2.629/8.631/2.313 ms
next, with this patch:
% iperf3-darwin -t 30 -c 172.31.2.93
...
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.00 sec 281 MBytes 78.6 Mbits/sec 763132 sender
[ 5] 0.00-30.21 sec 280 MBytes 77.9 Mbits/sec receiver
% iperf3-darwin -t 30 -c 172.31.2.93 -P 5
...
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.01 sec 106 MBytes 29.7 Mbits/sec 2057644 sender
[ 5] 0.00-30.18 sec 106 MBytes 29.4 Mbits/sec receiver
[ 7] 0.00-30.01 sec 55.7 MBytes 15.6 Mbits/sec 728640 sender
[ 7] 0.00-30.18 sec 55.5 MBytes 15.4 Mbits/sec receiver
[ 9] 0.00-30.01 sec 76.0 MBytes 21.2 Mbits/sec 295540 sender
[ 9] 0.00-30.18 sec 75.7 MBytes 21.1 Mbits/sec receiver
[ 11] 0.00-30.01 sec 82.4 MBytes 23.0 Mbits/sec 119032 sender
[ 11] 0.00-30.18 sec 81.8 MBytes 22.7 Mbits/sec receiver
[ 13] 0.00-30.01 sec 84.6 MBytes 23.7 Mbits/sec 588184 sender
[ 13] 0.00-30.18 sec 84.0 MBytes 23.3 Mbits/sec receiver
[SUM] 0.00-30.01 sec 405 MBytes 113 Mbits/sec 3789040 sender
[SUM] 0.00-30.18 sec 403 MBytes 112 Mbits/sec receiver
% iperf3-darwin -t 30 -c 172.31.3.1
...
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.00 sec 528 MBytes 148 Mbits/sec 32152 sender
[ 5] 0.00-30.16 sec 527 MBytes 147 Mbits/sec receiver
% iperf3-darwin -t 30 -c 172.31.3.1 -P 5
...
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.00 sec 284 MBytes 79.4 Mbits/sec 1448 sender
[ 5] 0.00-30.17 sec 284 MBytes 78.9 Mbits/sec receiver
[ 7] 0.00-30.00 sec 285 MBytes 79.6 Mbits/sec 0 sender
[ 7] 0.00-30.17 sec 285 MBytes 79.1 Mbits/sec receiver
[ 9] 0.00-30.00 sec 303 MBytes 84.8 Mbits/sec 0 sender
[ 9] 0.00-30.17 sec 303 MBytes 84.2 Mbits/sec receiver
[ 11] 0.00-30.00 sec 251 MBytes 70.1 Mbits/sec 0 sender
[ 11] 0.00-30.17 sec 250 MBytes 69.6 Mbits/sec receiver
[ 13] 0.00-30.00 sec 248 MBytes 69.5 Mbits/sec 0 sender
[ 13] 0.00-30.17 sec 248 MBytes 69.1 Mbits/sec receiver
[SUM] 0.00-30.00 sec 1.34 GBytes 383 Mbits/sec 1448 sender
[SUM] 0.00-30.17 sec 1.34 GBytes 381 Mbits/sec receiver
gw$ cd /usr/src/sys/arch/$(machine)/compile/GENERIC.MP
gw$ doas time make config
cd /usr/obj/sys/arch/octeon/compile/GENERIC.MP && config -s /usr/src/sys -b
/usr/src/sys/arch/octeon/compile/GENERIC.MP/obj
/usr/src/sys/arch/octeon/conf/GENERIC.MP
28.76 real 1.62 user 0.49 sys
gw$ doas time make config
cd /usr/obj/sys/arch/octeon/compile/GENERIC.MP && config -s /usr/src/sys -b
/usr/src/sys/arch/octeon/compile/GENERIC.MP/obj
/usr/src/sys/arch/octeon/conf/GENERIC.MP
32.63 real 1.53 user 0.65 sys
gw$
and ping from my laptop:
--- 172.31.2.1 ping statistics ---
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 9.951/76.632/204.249/61.828 ms
and ping from iperf machine:
--- 172.31.3.1 ping statistics ---
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 6.074/168.085/577.976/166.932 ms
something is very wrong here.
--
wbr, Kirill
