On 1/30/23 14:17, Paul Mather wrote:
TL;DR: When working from home, I can max out my residential 200 Mbit network
connection when downloading from remote Linux hosts at $JOB but only manage
about 20% of my max residential connection speed when downloading from remote
FreeBSD hosts at $JOB. When at $JOB, both FreeBSD and Linux hosts have no
problem saturating their GbE connections transferring between each other. Why
is this and how can I debug and fix it?
I have a 200 Mbit residential cable connection (Xfinity, 200 Mbit down/~10 Mbit
up). I've noticed recently that I can easily get 10--20 MB/s download speeds
when transferring data from Linux hosts at work but when I try to download that
same data from the FreeBSD hosts I use the speed usually tops out at 3--4 MB/s.
These are Linux and FreeBSD hosts that are on the same subnet at work.
Transfers from the FreeBSD hosts at work (within-subnet and within-site) are
fine and match those of the Linux hosts---often 112 MB/s. So, it just appears
to be the traffic over the WAN to my home that is affected. The WAN path from
home to this subnet is typically 15 hops with a typical average ping latency of
about 23 ms.
The FreeBSD hosts are a mixture of -CURRENT, 13-STABLE, and 13.1-RELEASE. I had done some TCP
tuning based upon the calomel.org <http://calomel.org/> tuning document
(https://calomel.org/freebsd_network_tuning.html), but removed those tuning settings when I
noticed the problem but the problem still persists. The only remaining customisation is that
the 13-STABLE has "net.inet.tcp.cc.algorithm=cubic". (I notice that -CURRENT now has
this as default so wanted to try that on 13-STABLE, too.) The FreeBSD systems are using either
igb or em NICs. The Linux systems are using similar hardware. None has a problem maintaining
local GbE transfer speeds---it's only the slower/longer WAN connections that have problems for
the FreeBSD hosts.
It seems that Linux hosts cope with the WAN path to my home better than the
FreeBSD systems. Has anyone else noticed this? Does anyone have any idea as
to what is obviously going wrong here and how I might debug/fix the FreeBSD
hosts to yield faster speeds? My workaround at the moment is to favour using
the remote Linux hosts for bulk data transfers. (I don't like this workaround.)
Any help/insight is gratefully appreciated.
Cheers,
Paul.
sysctl net.inet.tcp.cc.algorithm=htcp
I would set "htcp" on the server and home computer to improve through in
your type of situation.
