casaroli opened a new pull request, #22131:
URL: https://github.com/apache/nuttx/pull/22131

   ## Summary
   
   With `CONFIG_NET_SEND_BUFSIZE > 0`, a TCP `send()` that finds the send 
buffer full waits on `conn->snd_sem`. It compares the queued bytes with the 
limit while it holds the device and connection locks; then 
`conn_dev_sem_timedwait()` releases the locks and waits. 
`tcp_sendbuffer_notify()` posted the semaphore only when a thread already 
waited on it (count < 0). When the ACKs that empty the write queue are handled 
between the release of the locks and the wait, the post is lost. This happens 
when the work queue of a network driver waits for the device lock while the 
sender holds it (1 CPU), or when it runs on another CPU (SMP). If no other ACK 
comes, the sender waits until its `SO_SNDTIMEO` time-out, or for ever.
   
   A test-only log (not in this PR) in `tcp_sendbuffer_notify()` shows the lost 
post: with the sender between the release of the locks and the wait, the count 
is 0 and the notify does nothing. At the end of the 30 s wait of the sender, 
the write queue and the unacked queue are empty, nothing is unacked, the peer 
window is open, and the count is 0.
   
   This PR posts also when the count is 0. A count that no sender waited for 
only makes a sender check the limit once more. UDP (`udp_sendbuffer_notify()`) 
has the same code and the same window (the sender holds the connection lock 
only), and gets the same change in a second commit.
   
   ## Impact
   
   TCP and UDP with `CONFIG_NET_SEND_BUFSIZE > 0` (or a send buffer size from 
`SO_SNDBUF`): a send that waits on the send limit is woken when the buffer 
drains. With `CONFIG_NET_SEND_BUFSIZE` 0 (the default) this code is not 
compiled. No API, ABI or Kconfig change.
   
   Build modes and architectures: common network code, the same in flat, 
protected and kernel builds, with and without SMP.
   
   Not changed: the packet socket (`pkt_sendbuffer_notify()`) and CAN 
(`can_sendbuffer_notify()`) have the same `val < 0` check.
   
   ## Testing
   
   qemu-armv8a:knsh under HVF (Apple M-series), 96 IOBs of 196 bytes, 
`IOB_THROTTLE` 8, `CONFIG_NET_TCP_WRITE_BUFFERS`, 
`CONFIG_NET_SEND_BUFSIZE=2156` and `CONFIG_NET_RECV_BUFSIZE=2156`, master 
5e4ecd1d603. The programs run from `/system/bin`.
   
   - `tcpsndwake`: rounds of "send 16 KiB (or 64 KiB, 4 KiB) over 127.0.0.1, 
wait for a 1-byte reply from a thread that reads it" for 30 s, 5 s time-outs; 
FAIL when a round waits.
   - `tcpthru send`: 4 MiB to a host program over the QEMU user network, 8 
times in one boot; `SO_SNDTIMEO` 30 s.
   - `test_tcp_send_limit_wakeup`, a new case of `cmocka_net_tcp` 
(apache/nuttx-apps#APPS): the same rounds as `tcpsndwake` for 10 s, 2 s limit 
per round.
   
   The `cmocka_net_tcp` runs have #22025 on both sides: without it, the 
connections in TIME_WAIT of the earlier cases can use all devif callbacks (see 
below).
   
   | Test | master | this PR |
   |---|---|---|
   | `tcpsndwake`, 4 CPUs, 4 runs (16, 64, 4, 16 KiB) | FAIL in 3 of 4: a round 
waits 5 s, 10 s, 15 s | PASS 4 of 4 (2.6 million rounds) |
   | `tcpsndwake`, 1 CPU, 3 runs | PASS | |
   | `tcpthru send` to the host, 1 CPU, 8 uploads | 2 of 8 take 30 s (the 
time-out) | 0 of 8 |
   | `test_tcp_send_limit_wakeup`, 4 CPUs, 5 boots | FAIL 5 of 5: a round waits 
10 s or 15 s | PASS 5 of 5 |
   | `test_tcp_send_limit_wakeup`, 1 CPU | | PASS |
   | `test_tcp_send_limit_wakeup`, flat qemu-armv8a:nsh, 4 CPUs | PASS (1 boot) 
| PASS |
   | `test_tcp_send_limit_wakeup`, 4 CPUs, `CONFIG_NET_SEND_BUFSIZE` 0 | | PASS 
|
   | `ostest`, 1 and 4 CPUs, flat nsh | | status 0 |
   
   With the test-only log, the 1-CPU uploads on master waited 30 s in 1 and in 
4 of 8 runs (two boots; the log makes the window longer).
   
   The `ENOMEM` of `send()` that we saw after several transfers with 
`CONFIG_NET_SEND_BUFSIZE=8192` is not in this code: connections in TIME_WAIT 
keep three devif callbacks each for two minutes (the close callback, the 
disabled send callback and the monitor), and with the default pool of 16 a few 
closed connections use all of them. #22025 fixes that (its commit "Free the 
send callback and the monitor of a closed connection"); with #22025 the same 
sequence passes (8 transfers in one boot, `CONFIG_NET_SEND_BUFSIZE=8192`: 
master `ENOMEM` after the fifth, master with #22025 all pass).
   
   Builds in the NuttX CI image (`EXTRAFLAGS="-Wno-cpp -Werror"`), master and 
this PR: all OK, 0 warnings. Text size (bytes):
   
   | Configuration | master | this PR |
   |---|---|---|
   | qemu-armv8a:netnsh (no TCP write buffers, no send limit) | 554188 | 554172 
(-16) |
   | qemu-armv8a:netnsh + `NET_TCP_WRITE_BUFFERS` (no send limit) | 557164 | 
557148 (-16) |
   | the same + `NET_UDP_WRITE_BUFFERS` + `NET_SEND_BUFSIZE=8192` (TCP and UDP 
code built) | 559432 | 559424 (-8) |
   | qemu-armv8a:netnsh_smp + write buffers | 532864 | 532864 (+0) |
   | sim:tcpblaster | 997922 | 997922 (+0) |
   
   The first two rows do not build the changed code: all symbols have the same 
sizes, the 16 bytes are in unnamed read-only data.
   
   `tools/checkpatch.sh -c -u -m -g` passes. Documentation: no page describes 
this wait; no change.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to