casaroli opened a new pull request, #22078:
URL: https://github.com/apache/nuttx/pull/22078

   ## Summary
   
   Three defects of TCP retransmission, each in its own commit, and the 
documentation.
   
   1. **`net/tcp: Restart the retransmission timer only as RFC 6298 says.`** 
`tcp_fill_header()` restarted the retransmission timer for every outgoing 
segment, also for an ACK without data, and `tcp_input()` restarted it for every 
ACK that was not an old one, also for a duplicate ACK (with an RTT sample from 
it and a reset of `conn->nrtx`). While the peer sends data, each of its 
segments carries such an ACK and gets an ACK from us, so the timer did not 
expire until the peer stopped. RFC 6298, section 5: the timer starts when data 
is sent and restarts when an ACK acknowledges new data. Now only a segment that 
takes sequence space (data, SYN, FIN) restarts it when it is sent, and an ACK 
that is not beyond `snd_wl2` (the highest ACK so far) gives no RTT sample and 
leaves the timer, `sndseq` and `nrtx` alone (it still sets `TCP_ACKDATA`).
   2. **`net/tcp: Do not count a segment with data as a duplicate ACK.`** RFC 
5681, section 2: a duplicate ACK carries no data. A peer that sends data while 
our segment is in flight repeats the ACK number in each segment. 
`psock_send_eventhandler()` and NewReno's `tcp_cc_recv_ack()` counted those 
segments, so every third one started a fast retransmission. The fast 
retransmission for a segment with data also corrupted the stream: it notified 
the driver while the device held the received segment, the driver can poll at 
once in `netdev_txnotify_dev()`, and after that poll the code put back the 
buffer and the lengths but not `d_sndlen`. When the poll stopped right after a 
sent segment, the reply to the received segment sent those bytes as new data, 
and then the real data under the same sequence numbers (an SFTP upload over SSH 
failed with a bad MAC). Now only a segment without data is a duplicate ACK, and 
that code is removed.
   3. **`net/tcp: Double the retransmission timeout at the first expiry too.`** 
`tcp_timer()` computed `conn->rto = TCP_RTO << nrtx` before it counted the 
expiry, so the timeouts of one segment were 1.5 s, 1.5 s, 3 s, 6 s (with the 
default `NET_TCP_RTO`). RFC 6298, section 5.5. Now 1.5 s, 3 s, 6 s. 
`TCP_MAXRTX`, `TCP_MAXSYNRTX` and the largest shift do not change; the time 
until `TCP_MAXRTX` gives up grows from about 96 s to about 118 s.
   4. **`Documentation/os/networking: Describe the retransmission timer.`** The 
TCP state machine page.
   
   ## Impact
   
   TCP only (`net/tcp`, one call in `net/sixlowpan/sixlowpan_tcpsend.c`). No 
API, ABI or Kconfig change. `tcp_fill_header()` and `tcp_cc_recv_ack()` get one 
more argument; both are internal (`net/tcp/tcp.h`), and every caller is changed 
(`tcp_send.c`, `sixlowpan_tcpsend.c`, `tcp_input.c`).
   
   Behaviour on the wire: fewer retransmissions (no fast retransmission from 
segments with data, no ACK that keeps the timer from expiring) and the second 
timeout of a segment twice the first. A segment with data that repeats the ACK 
number no longer triggers a fast retransmission with `NET_TCP_FAST_RETRANSMIT` 
or NewReno; three duplicate ACKs without data still do.
   
   Build modes and options: common network code, the same in flat, protected 
and kernel builds and on every architecture. With and without 
`NET_TCP_WRITE_BUFFERS`, `NET_TCP_FAST_RETRANSMIT`, `NET_TCP_CC_NEWRENO`, 
`NET_TCP_SELECTIVE_ACK`, `NET_TCPURGDATA`, `NET_SENDFILE`, `NET_TCP_FIXED_RTO`: 
see the builds below. SMP: the changed code runs under the device and 
connection locks, as before.
   
   ## Testing
   
   qemu-armv8a:knsh (kernel build) under HVF (Apple M-series), TCP over the 
loopback device and a virtio-net NIC that QEMU loops back to itself (`-netdev 
dgram` to its own port), 64 IOBs of 1534 bytes, write buffers, dynamic 
connections and callbacks. qemu-armv8a:nsh (flat build) with the default pools. 
Master is 84abb4756d3 (one 4-CPU master boot: 1194bc8c4ff, no network change 
between them); the tests are standalone programs run from `/system/bin`. No 
riscv64 or x86_64 run here. The runs and the sizes below are of an earlier 
version of this PR that also had a commit for the receive window update (RFC 
1122, section 4.2.3.3); that commit is not in this PR.
   
   - `dupackdata`: a scripted TCP peer on a packet socket of the looped-back 
NIC, in the same guest. Per round: the handshake with a one-segment window; the 
server writes 32 KiB; the peer sends 40 data segments 0.1 s apart that repeat 
the old ACK (as if our first segment were lost), then ACKs everything. PASS 
needs 1 or 2 timeout retransmissions of the first segment in those 4 s, none 
within 50 ms of a data segment of the peer (a fast retransmission), the stream 
byte-exact and no sequence number with two payloads. `--backoff`: 60 segments 
over 6 s; the second timeout must be at least 1.6 times the first.
   - `tcpbulk bulk`: 3 x 8 MiB over 127.0.0.1, 20 s each. `tcpbulk zwin`: 3 x 4 
MiB where the reader starts after 1 s (the window fills, the sender's write 
buffers take the free IOBs, then the window must open again).
   - `tcpeof 1024 10 p|s`: 1 MiB in 16 KiB `send()`s or in one `send()`, then 
`close()`; the reader must get every byte and then end-of-file, 10 times.
   - `cmocka_net_tcp` (nuttx-apps) and `ostest`.
   
   | Test | master | this PR |
   |---|---|---|
   | knsh 1 CPU: `dupackdata` 5 rounds, twice | FAIL in round 0: 7 fast 
retransmissions of the first segment in 4 s | 10 of 10 rounds pass: 1 timeout 
retransmission, no fast one, stream byte-exact |
   | knsh 1 CPU: `dupackdata --backoff` 3 rounds (2 boots) | FAIL in round 0: 7 
fast retransmissions | 6 of 6 rounds pass |
   | knsh 4 CPUs (`CONFIG_SMP`): `dupackdata` 5 rounds twice, `--backoff` 3 
rounds (2 boots) | FAIL in round 0: 7 fast retransmissions | 16 of 16 rounds 
pass, no fast retransmission |
   | knsh 1 CPU: `tcpbulk bulk`, `tcpbulk zwin` x3, `tcpeof` p and s | pass | 
pass |
   | knsh 1 and 4 CPUs, nsh: `cmocka_net_tcp` | 6 of 7 (`test_tcp_connect_ipv4` 
fails) | the same |
   | knsh 1 and 4 CPUs, nsh: `ostest` | status 0 | status 0 |
   
   TCP over 127.0.0.1 with 4 CPUs: a reader sometimes gets `ECONNRESET` or a 
transfer stalls, on master and with this PR at about the same rate. 4 CPUs, 5 
boots each: `tcpeof` 6 of 130 iterations fail on master, 5 of 120 with this PR; 
`tcpbulk bulk` 6 of 10 runs fail on master, 4 of 8 with this PR; `tcpbulk zwin` 
3 of 11 and 2 of 9; 2 and 1 boots stop in a hang after a transfer. With 
`CONFIG_DEBUG_NET_WARN` the reset is `psock_send_eventhandler: WARNING: 
Expiring wrb ... nrtx=8`: a write buffer that the loopback device dropped (no 
free IOB at the receiver) reaches `TCP_MAXRTX` retransmissions and the 
connection is reset. The count of a write buffer is not reset when an ACK takes 
part of it. That is not changed here.
   
   `test_tcp_connect_ipv4` of `cmocka_net_tcp` fails on master and with this PR 
alike (its loopback server refuses the connection).
   
   Builds in the NuttX CI image (`EXTRAFLAGS="-Wno-cpp -Werror"`), master and 
this PR: all OK, 0 warnings. Text size (bytes):
   
   | Configuration | Build mode | master | this PR |
   |---|---|---|---|
   | qemu-armv8a:netnsh (no write buffers) | flat | 549408 | 549464 (+56) |
   | qemu-armv8a:netnsh + `NET_TCP_WRITE_BUFFERS`, `NET_SOLINGER` | flat | 
552380 | 552380 (+0) |
   | the same, CMake | flat | 553164 | 553164 (+0) |
   | qemu-armv8a:netnsh_smp + write buffers | flat, SMP | 528000 | 528000 (+0) |
   | qemu-armv8a:netnsh + write buffers + `NET_TCP_CC_NEWRENO` | flat | 553056 
| 553068 (+12) |
   | qemu-armv8a:netnsh + `NET_TCP_CC_NEWRENO`, no write buffers | flat | 
550068 | 550136 (+68) |
   | qemu-armv8a:netnsh + write buffers, `NET_TCP_FAST_RETRANSMIT` off | flat | 
552124 | 552184 (+60) |
   | qemu-armv8a:netnsh + write buffers, NewReno, out-of-order, SACK, urgent 
data, sendfile, delayed ACK | flat | 557188 | 557208 (+20) |
   | the same without write buffers and NewReno, `NET_TCP_FIXED_RTO` | flat | 
552740 | 552804 (+64) |
   | rv-virt:pnsh64 + TCP, write buffers | protected | 184672 | 184864 (+192) |
   | rv-virt:pnsh64 + TCP, no write buffers | protected | 182512 | 182512 (+0) |
   | rv-virt:knetnsh64 | kernel | 229123 | 229161 (+38) |
   | rv-virt:knetnsh64 + write buffers, dynamic pools | kernel | 231501 | 
231479 (-22) |
   | lc823450-xgevk:krndis | protected | 193228 | 193284 (+56) |
   | sim:tcpblaster | sim | 997810 | 997698 (-112) |
   | sim:tcpblaster without write buffers | sim | 989122 | 989266 (+144) |
   | sim:sixlowpan | sim, 6LoWPAN | 637756 | 637676 (-80) |
   | samv71-xult:mrf24j40-starhub | flat, 6LoWPAN | 244864 | 244832 (-32) |
   | lm3s6432-s2e:nsh (96 KiB flash) | flat | 64764 | 64788 (+24) |
   
   `tools/checkpatch.sh -c -u -m -g` passes. Documentation: commit 4.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to