Hi Stuart
Thank you very much for the link.
The total ssh based performace depends strongly on the server hardware
(and installed OSes).
For the "fastest" test configuration (server hardware / installed OS) I
was possible to achieve
a total trasfer speed of approx 400MBytes/s (on the 10Gbit fiber link)
for few parallel read/write scp sessions.
I don't know if the ix network drivers used in FreeBSD 13.0 orLinux
kernel are much more efficient
and it is possible to use the ix network driver from FreeBSD 13.0 in
OpenBSD 6.4.
# ---
# fbsdsrv1 kernel: CPU: Intel(R) Xeon(R) CPU X5690 @ 3.47GHz
(3465.76-MHz K8-class CPU)
# fbsdsrv1 kernel: FreeBSD/SMP: Multiprocessor System Detected: 24 CPUs
# fbsdsrv1 kernel: real memory = 103079215104 (98304 MB)
# fbsdsrv1 kernel: da0: <HP RAID ADG OK> Fixed Direct Access SPC-3 SCSI
device
# fbsdsrv1 kernel: da0: Serial Number PACCRCN80ZL1RD7
# fbsdsrv1 kernel: da0: 135.168MB/s transfers
# fbsdsrv1 kernel: da0: Command Queueing enabled
# fbsdsrv1 kernel: da0: 839893MB (1720102192 512 byte sectors)
# fbsdsrv1 kernel: ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver>
port 0x7000-0x701f mem 0xe9a80000-0xe9afffff,0xe9a70000-0xe9a73fff irq 26
# at device 0.0 on pci6
# fbsdsrv1 kernel: ix0: Using 2048 tx descriptors and 2048 rx descriptors
# fbsdsrv1 kernel: ix0: queue equality override not set, capping
rx_queues at 12 and tx_queues at 12
# fbsdsrv1 kernel: ix0: Using 12 rx queues 12 tx queues
# fbsdsrv1 kernel: ix0: Using MSI-X interrupts with 13 vectors
# fbsdsrv1 kernel: ix0: allocated for 12 queues
# fbsdsrv1 kernel: ix0: allocated for 12 rx queues
# fbsdsrv1 kernel: ix0: Ethernet address: 90:e2:ba:16:20:a4
# fbsdsrv1 kernel: ix0: PCI Express Bus: Speed 5.0GT/s Width x4
# fbsdsrv1 kernel: ix0: netmap queues/slots: TX 12/2048, RX 12/2048
# fbsdsrv1 kernel: ix1: <Intel(R) PRO/10GbE PCI-Express Network Driver>
port 0x7020-0x703f mem 0xe9980000-0xe99fffff,0xe9970000-0xe9973fff irq 25
# at device 0.1 on pci6
# fbsdsrv1 kernel: ix1: Using 2048 tx descriptors and 2048 rx descriptors
# fbsdsrv1 kernel: ix1: queue equality override not set, capping
rx_queues at 12 and tx_queues at 12
# fbsdsrv1 kernel: ix1: Using 12 rx queues 12 tx queues
# fbsdsrv1 kernel: ix1: Using MSI-X interrupts with 13 vectors
# fbsdsrv1 kernel: ix1: allocated for 12 queues
# fbsdsrv1 kernel: ix1: allocated for 12 rx queues
# fbsdsrv1 kernel: ix1: Ethernet address: 90:e2:ba:16:20:a5
# fbsdsrv1 kernel: ix1: PCI Express Bus: Speed 5.0GT/s Width x4
# fbsdsrv1 kernel: ix1: netmap queues/slots: TX 12/2048, RX 12/2048
# ---
ironm@fbsdsrv1:~ $ scp t1.iso
ironm@200.0.0.20:/home/ironm/fbsd2fbs-send-conf1.iso
Password for ironm@fbsdsrv2:
t1.iso 100% 3626MB 132.0MB/s 00:27
Kind regards
Mark
--
m...@it-infrastrukturen.org
Am 09.04.2019 13:31, schrieb Stuart Henderson:
On 2019-04-07, Mark Schneider <m...@it-infrastrukturen.org> wrote:
Short feedback:
Just for the test I have checked the 10GBit network performance
between two FreeBSD 13.0 servers (both HP DL380g7 machines)
transfering data in both directions
# ---
ironm@fbsdsrv2:~ $ scp ironm@200.0.0.10:/home/ironm/t2.iso t100.iso
Password for ironm@fbsdsrv1:
t2.iso 100% 3626MB 130.2MB/s 00:27
# ---
ironm@fbsdsrv2:~ $ scp obsd2fbsd.iso ironm@200.0.0.10:/home/ironm/t1.iso
Password for ironm@fbsdsrv1:
obsd2fbsd.iso 100% 3626MB 140.4MB/s 00:25
# ---
scp is a *terrible* way to test network performance. If you are only
interested in scp performance between two hosts then it's relevant,
and you can probably improve speeds by using something other than scp.
Otherwise irrelevant.
The ssh performance using 10GBit network connection on FreeBSD 13.0
is approx 7 times higher than the one on OpenBSD 6.4.
Is it the question of the "ix" NIC driver of OpenBSD 6.4?
(X520-DA2 NICs from Intel)
Does one of you achieve good 10Gbit network performance with other
10Gbit NICs?
FreeBSD's network stack can make better use of multiple processors.
OpenBSD is improving (you can read some stories about this work at
www.grenadille.net) but is slower.
Jumbo frames should help if you can use them. Much of network
performance is related to packets-per-second not bits-per-second.
For scp, switching ciphers/MACs is likely to speed things up too.
