On Jan 30, 2023, at 4:59 PM, Marek Zarychta <[email protected]> wrote:
> W dniu 30.01.2023 o 22:17, Paul Mather pisze: >> TL;DR: When working from home, I can max out my residential 200 Mbit network >> connection when downloading from remote Linux hosts at $JOB but only manage >> about 20% of my max residential connection speed when downloading from >> remote FreeBSD hosts at $JOB. When at $JOB, both FreeBSD and Linux hosts >> have no problem saturating their GbE connections transferring between each >> other. Why is this and how can I debug and fix it? >> >> I have a 200 Mbit residential cable connection (Xfinity, 200 Mbit down/~10 >> Mbit up). I've noticed recently that I can easily get 10--20 MB/s download >> speeds when transferring data from Linux hosts at work but when I try to >> download that same data from the FreeBSD hosts I use the speed usually tops >> out at 3--4 MB/s. These are Linux and FreeBSD hosts that are on the same >> subnet at work. Transfers from the FreeBSD hosts at work (within-subnet and >> within-site) are fine and match those of the Linux hosts---often 112 MB/s. >> So, it just appears to be the traffic over the WAN to my home that is >> affected. The WAN path from home to this subnet is typically 15 hops with a >> typical average ping latency of about 23 ms. >> >> The FreeBSD hosts are a mixture of -CURRENT, 13-STABLE, and 13.1-RELEASE. I >> had done some TCP tuning based upon the calomel.org <http://calomel.org/> >> tuning document (https://calomel.org/freebsd_network_tuning.html), but >> removed those tuning settings when I noticed the problem but the problem >> still persists. The only remaining customisation is that the 13-STABLE has >> "net.inet.tcp.cc.algorithm=cubic". (I notice that -CURRENT now has this as >> default so wanted to try that on 13-STABLE, too.) The FreeBSD systems are >> using either igb or em NICs. The Linux systems are using similar hardware. >> None has a problem maintaining local GbE transfer speeds---it's only the >> slower/longer WAN connections that have problems for the FreeBSD hosts. >> >> It seems that Linux hosts cope with the WAN path to my home better than the >> FreeBSD systems. Has anyone else noticed this? Does anyone have any idea >> as to what is obviously going wrong here and how I might debug/fix the >> FreeBSD hosts to yield faster speeds? My workaround at the moment is to >> favour using the remote Linux hosts for bulk data transfers. (I don't like >> this workaround.) >> >> Any help/insight is gratefully appreciated. >> >> Cheers, >> >> Paul. >> > While playing with different mod_cc(4) might bring some improvement, to get a > real boost I'd suggest enabling tcp_rack(4) if feasible. I am interested in trying this out, but believe it is more feasible in my case for the -STABLE and -CURRENT systems I am using, not so much for the -RELEASE systems that are kept up to date via binary freebsd-update updates. My reading of the tcp_rack(4) man page is that you have to build a custom kernel as, unlike the cc_* congestion control algorithms, the loadable tcp_rack module is not built by default. Is that an accurate reading? Cheers, Paul.
