On Wed, Nov 19, 2025 at 11:24:01AM -0500, Cheng Cui wrote:
That's good news! :)
I did some very general tests: first ones were one vm upgraded over
several iterations. This vm being enc zfs mounted as a zvol
on a raidz1 zpool (spinning rust). Nothing changed apart from the OS version
The tl;dr is that 16.0-CURRENT ufs2 nodebug kernel & userland is the clear
winner in terms of throughput; it is almost equal (941 Mbits/sec) to line level.
freebsd 14.3-p3
===============
iperf3 -B 192.168.1.121 -c 192.168.1.230 -l 1M -t 30 -i 2 ; netstat -nCRp tcp | egrep
"Active|Proto|5201"
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.01 sec 1.38 GBytes 396 Mbits/sec 0 sender
[ 5] 0.00-30.01 sec 1.38 GBytes 396 Mbits/sec receiver
stable/15-n281161-fc22812b6685
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.03 sec 1.57 GBytes 450 Mbits/sec 0 sender
[ 5] 0.00-30.03 sec 1.57 GBytes 450 Mbits/sec receiver
16.0-CURRENT
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.13 sec 1.77 GBytes 506 Mbits/sec 0 sender
[ 5] 0.00-30.13 sec 1.77 GBytes 506 Mbits/sec receiver
====================================================
These next few tests exploring differences whether vm on
zpool host was ufs backed, zfs backed, zvol backed or zvol with compression
whether these factors influenced throughput.
all 16.0-current
16.0-CURRENT ufs2 nodebug kernel & userland
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.05 sec 3.28 GBytes 939 Mbits/sec 0 sender
[ 5] 0.00-30.05 sec 3.28 GBytes 938 Mbits/sec receiver
16-current ufs2 installed the new way - debug kernel & userland
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.07 sec 1.88 GBytes 537 Mbits/sec 0 sender
[ 5] 0.00-30.09 sec 1.88 GBytes 537 Mbits/sec receiver
as above, -rxcsum -txcsum -tso -tso6 -lro
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.16 sec 1.46 GBytes 417 Mbits/sec 0 sender
[ 5] 0.00-30.15 sec 1.46 GBytes 417 Mbits/sec receiver
as above, nodebug kernel
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.13 sec 3.01 GBytes 858 Mbits/sec 0 sender
[ 5] 0.00-30.15 sec 3.01 GBytes 858 Mbits/sec receiver
as above, nodebug kernel & world
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.01 sec 3.08 GBytes 881 Mbits/sec 0 sender
[ 5] 0.00-30.02 sec 3.08 GBytes 881 Mbits/sec receiver
16-current installed the traditional way - zfs & debug kernel
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.03 sec 2.70 GBytes 772 Mbits/sec 0 sender
[ 5] 0.00-30.03 sec 2.70 GBytes 771 Mbits/sec receiver
as above, -rxcsum -txcsum -tso -tso6 -lro
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.05 sec 2.68 GBytes 765 Mbits/sec 0 sender
[ 5] 0.00-30.05 sec 2.68 GBytes 765 Mbits/sec receiver
as above, nodebug kernel
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.02 sec 2.92 GBytes 835 Mbits/sec 0 sender
[ 5] 0.00-30.03 sec 2.92 GBytes 835 Mbits/sec receiver
as above, nodebug kernel & userland
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.04 sec 2.96 GBytes 846 Mbits/sec 0 sender
[ 5] 0.00-30.04 sec 2.96 GBytes 846 Mbits/sec receiver
16-current installed the traditional way - zfs & debug kernel -
fileimage, encrypted zfs
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.04 sec 1.90 GBytes 544 Mbits/sec 0 sender
[ 5] 0.00-30.04 sec 1.90 GBytes 544 Mbits/sec receiver
as above, -rxcsum -txcsum -tso -tso6 -lro
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.11 sec 1.79 GBytes 510 Mbits/sec 0 sender
[ 5] 0.00-30.12 sec 1.79 GBytes 509 Mbits/sec receiver
nodebug kernel -rxcsum -txcsum -tso -tso6 -lro
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.01 sec 2.89 GBytes 826 Mbits/sec 0 sender
[ 5] 0.00-30.02 sec 2.89 GBytes 826 Mbits/sec receiver
as above, offloading enabled
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.01 sec 2.88 GBytes 824 Mbits/sec 0 sender
[ 5] 0.00-30.02 sec 2.88 GBytes 824 Mbits/sec receiver
nodebug enccrypted on zvol
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.09 sec 1.86 GBytes 530 Mbits/sec 0 sender
[ 5] 0.00-30.10 sec 1.86 GBytes 530 Mbits/sec receiver
as above, -rxcsum -txcsum -tso -tso6 -lro
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.08 sec 1.60 GBytes 456 Mbits/sec 0 sender
[ 5] 0.00-30.08 sec 1.60 GBytes 456 Mbits/sec receiver
as above, offloading enabled
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.02 sec 2.26 GBytes 646 Mbits/sec 0 sender
[ 5] 0.00-30.03 sec 2.26 GBytes 646 Mbits/sec receiver
--
