Re: Too aggressive TCP ACKs
You can also think about MacOS’s delayed ACK setup in default is conservative. https://developer.apple.com/forums/thread/716394 -- Cheng Cui From: owner-freebsd-...@freebsd.org on behalf of Zhenlei Huang Date: Friday, October 21, 2022 at 11:01 AM To: Michael Tuexen Cc: freebsd-net@freebsd.org Subject: Re: Too aggressive TCP ACKs NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe. On Oct 21, 2022, at 10:34 PM, Michael Tuexen mailto:michael.tue...@lurchi.franken.de>> wrote: On 21. Oct 2022, at 16:19, Zhenlei Huang mailto:zlei.hu...@gmail.com>> wrote: Hi, While I was repeating https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=258755, I observed a strange behavior. The TCP ACKs from FreeBSD host are too aggressive. My setup is simple: A B [ MacOS ] <> [ FreeBSD VM ] 192.168.120.1192.168.12.134 (disable tso and lro) While A <--- B, i.e. A as server and B as client, the packets rate looks good. One session on B: root@:~ # iperf3 -c 192.168.120.1 -b 10m Connecting to host 192.168.120.1, port 5201 [ 5] local 192.168.120.134 port 54459 connected to 192.168.120.1 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.25 MBytes 10.5 Mbits/sec0257 KBytes [ 5] 1.00-2.00 sec 1.25 MBytes 10.5 Mbits/sec0257 KBytes [ 5] 2.00-3.00 sec 1.12 MBytes 9.44 Mbits/sec0257 KBytes [ 5] 3.00-4.00 sec 1.25 MBytes 10.5 Mbits/sec0257 KBytes [ 5] 4.00-5.00 sec 1.12 MBytes 9.44 Mbits/sec0257 KBytes [ 5] 5.00-6.00 sec 1.25 MBytes 10.5 Mbits/sec0257 KBytes [ 5] 6.00-7.00 sec 1.12 MBytes 9.44 Mbits/sec0257 KBytes [ 5] 7.00-8.00 sec 1.25 MBytes 10.5 Mbits/sec0257 KBytes [ 5] 8.00-9.00 sec 1.12 MBytes 9.44 Mbits/sec0257 KBytes [ 5] 9.00-10.00 sec 1.25 MBytes 10.5 Mbits/sec0257 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 12.0 MBytes 10.1 Mbits/sec0 sender [ 5] 0.00-10.00 sec 12.0 MBytes 10.1 Mbits/sec receiver iperf Done. Another session on B: root@:~ # netstat -w 1 -I vmx0 input vmx0 output packets errs idrops bytespackets errs bytes colls 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 342 0 0 22600526 0 775724 0 150 0 0 9900851 01281454 0 109 0 0 7194901 01357850 0 126 0 0 8316828 01246632 0 122 0 0 8052910 01370780 0 109 0 0 7194819 01233702 0 120 0 0 7920910 01370780 0 110 0 0 7260819 01233702 0 123 0 0 8118910 01370780 0 109 0 0 7194819 01233702 0 73 0 0 5088465 0 686342 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 While A ---> B, i.e. A as client and B as server, the ACKs sent from B looks strange. Session on A: % iperf3 -c 192.168.120.134 -b 10m Connecting to host 192.168.120.134, port 5201 [ 5] local 192.168.120.1 port 52370 connected to 192.168.120.134 port 5201 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 1.25 MBytes 10.5 Mbits/sec [ 5] 1.00-2.00 sec 1.25 MBytes 10.5 Mbits/sec [ 5] 2.00-3.00 sec 1.12 MBytes 9.44 Mbits/sec [ 5] 3.00-4.00 sec 1.25 MBytes 10.5 Mbits/sec [ 5] 4.00-5.00 sec 1.12 MBytes 9.44 Mbits/sec [ 5] 5.00-6.00 sec 1.25 MBytes 10.5 Mbits/sec [ 5] 6.00-7.00 sec 1.12 MBytes 9.44 Mbits/sec [ 5] 7.00-8.00 sec 1.25 MBytes 10.5 Mbits/sec [ 5] 8.00-9.00 sec 1.12 MBytes 9.44 Mbits/sec [ 5] 9.00-10.00 sec 1.25 MBytes 10.5 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.00 sec 12.0 MBytes 10.1 Mbits/sec sender [ 5] 0.00-10.00 sec 12.0 MBytes 10.1 Mbits/sec receiver iperf Done. Session on B: root@:~ # netstat -w 1 -I vmx0 input vmx0 output packets errs idrops bytespackets errs bytes colls 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 649 0 0
Re: why not enable tcp_pmtud_blackhole_detect in default
Kevin, Thanks for the reply. I think you have answered most of my doubts. But I would like to be clear that, is there a very strong reason why you run this in production even it often causes unnecessary lower MSS with lower performance? Thanks, Cheng Cui On 3/7/18, 3:35 PM, "Kevin Bowling" <kevin.bowl...@kev009.com> wrote: Cheng, We run this in production at Limelight Networks (i.e toward a broad spectrum of Internet hosts) and must to deal with some uncommon network topology. There are currently some limitations as you point out. Like you say the signaling is not perfect and we do often clamp MSS unnecessarily. There is also no probing to see if we can expand the MSS later. I think those issues should be fixed up before it's enabled by default and I don't know anyone working on it at the moment. Regards, On Wed, Mar 7, 2018 at 8:35 AM, Cui, Cheng <cheng@netapp.com> wrote: > Dear all, > > Reading through the tcp blackhole detection code (support RFC 4821) in FreeBSD including the recent bug fixes, I am wondering why is it still not enabled in default? Given the fact that this implementation was a merge from xnu, and the xnu has enabled it in default, do we have a plan to enable it in default? Or is there any concern about the side-effect from it as performance regression against some false positive blackhole event like a temporary link flap, which is long enough to trigger a lower MSS but shorter than 6 RTO? > > https://opensource.apple.com/source/xnu/xnu-1456.1.26/bsd/netinet/tcp_timer.c.auto.html << enabled in macOS 10.6 > https://reviews.freebsd.org/rS322967 << bug fixes > https://reviews.freebsd.org/rS272720 << merge from xnu > > Thanks, > --Cheng Cui > NetApp Scale Out Networking > https://netapp-meeting.webex.com/meet/chengc > > > ___ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org" ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
why not enable tcp_pmtud_blackhole_detect in default
Dear all, Reading through the tcp blackhole detection code (support RFC 4821) in FreeBSD including the recent bug fixes, I am wondering why is it still not enabled in default? Given the fact that this implementation was a merge from xnu, and the xnu has enabled it in default, do we have a plan to enable it in default? Or is there any concern about the side-effect from it as performance regression against some false positive blackhole event like a temporary link flap, which is long enough to trigger a lower MSS but shorter than 6 RTO? https://opensource.apple.com/source/xnu/xnu-1456.1.26/bsd/netinet/tcp_timer.c.auto.html << enabled in macOS 10.6 https://reviews.freebsd.org/rS322967 << bug fixes https://reviews.freebsd.org/rS272720 << merge from xnu Thanks, --Cheng Cui NetApp Scale Out Networking https://netapp-meeting.webex.com/meet/chengc ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: question about if a recent Linux patch on window scaling is required in FreeBSD
Add freebsd-net to increase the size of audience.
Thanks,
--Cheng Cui
NetApp Scale Out Networking
On 8/30/16, 10:19 AM, "Cui, Cheng" <cheng@netapp.com> wrote:
Refresh this question. Can anyone make a comment?
Thanks,
--Cheng Cui
NetApp Scale Out Networking
On 8/25/16, 3:52 PM, "Cui, Cheng" <cheng@netapp.com> wrote:
Hello everyone,
I hope this email could reach you well, because I found related
discussions about this topic on window scaling and the case of window
shrinking (or retraction or loss of precision). And I try to make this
question simple.
There is a recent Linux patch at receiver side to round-up advertised
window due to precision loss of window scaling. It reaches my attention
because the same problem could also happen between a pair of Linux and
FreeBSD nodes, and I am not aware of any similar patch in FreeBSD yet.
The Linux patch is this:
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=6
07bfbf2d55dd1cfe5368b41c2a81a8c9ccf4723
And I quote some description of the Linux patch below:
> If the sender uses up the entire window before it is shrunk, this can
> have chaotic effects on the connection. When sending ACKs,
> tcp_acceptable_seq() will notice that the window has been shrunk since
> tcp_wnd_end() is before tp->snd_nxt, which makes it choose
tcp_wnd_end()
>as
> sequence number. This will fail the receivers checks in tcp_sequence()
>however
> since it is before it's tp->rcv_wup, making it respond with a dupack.
I think the Linux's behavior is right ("ACK-only packets should be sent
with the largest in-window sequence number that has ever been sent."
ref:
https://www.ietf.org/mail-archive/web/tcpm/current/msg10512.html), it
actually chooses "tp->snd_una+tp->snd_wnd" (tcp_wnd_end()) instead of
tp->snd_nxt, as it thought tp->snd_nxt is out of window, in case of
precision loss which made the receiver's advertise-window smaller. But
at
the
other side, if the other side is FreeBSD, I think FreeBSD will also fail
the
check since "tp->snd_una+tp->snd_wnd" is before it's tp->rcv_nxt, and
ignore
the sequence number in the packet.
I also sent an email to t...@ietf.org asking if this Linux patch is RFC
7323
(window scaling part) compliant, but I have not get any reply yet.
So my question here is: Is there any recent change in FreeBSD to
accommodate the
Linux behavior ("tp->snd_una+tp->snd_wnd" as sequence number)? If not,
do
we
consider to apply the same way as in the Linux patch?
Thanks and apologize in advance if I did not do enough research,
--Cheng Cui
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
critical typos in log of Revision 252781
Hello, I sent this notification to Andre directly before. But I have not received any response. Hope this email again can reach someone in the email list to correct these typos. Log Message: MFC r291296, r291297, r291393 <== should be "MFC r251296, r251297, r251393" "Allow drivers to specify a maximum TSP length in bytes" should be "TSO length". Thanks, --Cheng Cui NetApp Scale Out Networking ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
