PR #24886 opened by dcyruschow URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24886 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24886.patch
# Summary of changes `udp_close()` stops the circular buffer RX thread with `pthread_cancel()` and then waits in `pthread_join()`. The RX thread keeps cancellation disabled while it handles each packet and enables it only around `recvfrom()`, relying on `recvfrom()` being a cancellation point. On Darwin, a `pthread_cancel()` that arrives while the target has cancellation disabled only marks the request as pending in user space. Neither re-enabling cancellation nor any later `recvfrom()` acts on it, so the request is lost and the thread never exits: it keeps reading while packets arrive, and blocks in `recvfrom()` once they stop. `udp_close()` then waits in `pthread_join()` forever. The request is lost only if it arrives while the thread handles a packet, so closing an input that is receiving data often hangs, while closing an idle input does not. This patch calls `pthread_testcancel()` after cancellation is enabled again. It acts on a request that arrived while cancellation was disabled; a request that arrives later reaches the thread while cancellation is enabled and interrupts `recvfrom()` as before. Win32 does not use `pthread_cancel()` here (`udp_close()` uses `shutdown()` + `CancelIoEx()`), and `compat/w32pthreads.h` has no `pthread_testcancel()`, so the call is skipped there. ### Why Darwin loses the request Apple's `pthread_setcancelstate(3)` says that requests made while cancelability is disabled "are held pending", and that with deferred cancellation they are "held pending until a cancellation point (see below) is reached". It lists `recvfrom()` as a cancellation point. The implementation does not do this for requests that arrive while cancellation is disabled: - `pthread_cancel()` always sets `_PTHREAD_CANCEL_PENDING` in user space, but asks the kernel to mark the thread (`__pthread_markcancel()`) only `if (state & PTHREAD_CANCEL_ENABLE)` ([libpthread-539.100.4 `src/pthread_cancelable.c` L96-L113](https://github.com/apple-oss-distributions/libpthread/blob/libpthread-539.100.4/src/pthread_cancelable.c#L96-L113)). - `pthread_setcancelstate(PTHREAD_CANCEL_ENABLE)` calls `__pthread_canceled(1)` and updates the user-space bits ([L119-L142](https://github.com/apple-oss-distributions/libpthread/blob/libpthread-539.100.4/src/pthread_cancelable.c#L119-L142)). In the kernel, action 1 only clears `UT_CANCELDISABLE`; it does not look at the user-space pending bit ([xnu-12377.121.6 `bsd/kern/kern_sig.c` L970-L998](https://github.com/apple-oss-distributions/xnu/blob/xnu-12377.121.6/bsd/kern/kern_sig.c#L970-L998)). - `recvfrom()` checks only the kernel mark: it calls `__pthread_testcancel(1)`, which returns `EINTR` only when `UT_CANCEL` is set ([`bsd/kern/uipc_syscalls.c` L2295-L2300](https://github.com/apple-oss-distributions/xnu/blob/xnu-12377.121.6/bsd/kern/uipc_syscalls.c#L2295-L2300), [`kern_sig.c` L913-L932](https://github.com/apple-oss-distributions/xnu/blob/xnu-12377.121.6/bsd/kern/kern_sig.c#L913-L932)). - `pthread_testcancel()` checks the user-space bits (`PTHREAD_CANCEL_ENABLE | _PTHREAD_CANCEL_PENDING`) and exits the thread ([L166-L196](https://github.com/apple-oss-distributions/libpthread/blob/libpthread-539.100.4/src/pthread_cancelable.c#L166-L196)), so it is the call that acts on the pending request. - A request made while cancellation is enabled sets the sticky kernel `UT_CANCEL` mark, interrupting a blocked `recvfrom()` or failing the next one on entry ([`kern_sig.c` L936-L963](https://github.com/apple-oss-distributions/xnu/blob/xnu-12377.121.6/bsd/kern/kern_sig.c#L936-L963)), so no window remains after the `pthread_testcancel()` call. The same `pthread_cancel()` logic is already in [libpthread-105.1.4](https://github.com/apple-oss-distributions/libpthread/blob/libpthread-105.1.4/src/pthread_cancelable.c#L103-L110), so this is long-standing Darwin behaviour, not a recent regression. ### Apple Feedback Assistant bug report I'm posting a report to Apple on this but since it's longstanding I don't expect it to be fixed soon and having this patch will workaround needing to upgrade macOS should they address the root cause. ### Testing Two programs are attached. **[udp_close_e2e.c](/attachments/83f935b7-59e0-4420-8006-7caf5aeb600d)** (needs libavformat): it opens a `udp://127.0.0.1:<port>` input with `avio_open2()`, reads while a local sender floods the port, and closes it, 200 cycles per run, with a 5 s watchdog on each cycle. Results on macOS 26, arm64, current master: | Build | Result | |---|---| | master | hang in `udp_close()` in 5/5 runs (cycles 10, 1, 1, 3, 3); a sample dump [sample_hang.txt](/attachments/c1321027-f556-413c-8bb6-fd50f6c36e20) of a hung run shows the RX thread still looping over `recvfrom()` and the fifo write while the sender keeps sending | | master + this patch | 10/10 runs complete all 200 cycles | **[pending_cancel_lost.c](/attachments/c3a7b65c-c369-4a8e-9283-b152b035033b)** (no FFmpeg, no sockets): it puts the cancel request into one exact window with a handshake, then blocks in `read()` on a pipe that never gets data. It is deterministic, and it shows the OS behaviour without FFmpeg: | Mode | Where the request arrives | macOS 26.7 (25G229), arm64 | Linux, glibc 2.41 | |---|---|---|---| | (none) | cancellation disabled, then enable + `read()` | **FAIL 3/3** (blocked in `read()`) | PASS 3/3 | | `testcancel` | same, with `pthread_testcancel()` before `read()` | PASS 3/3 | PASS 3/3 | | `enabled` | while blocked in `read()` (control) | PASS 3/3 | PASS 3/3 | | `gap` | cancellation enabled, after `pthread_testcancel()`, before `read()` | PASS 3/3 | PASS 3/3 | macOS: xnu-12377.161.15, Apple clang 21.0.0. Linux: `gcc:latest` Docker container on the same Mac (kernel 7.0.12-linuxkit, aarch64; Debian glibc 2.41-12+deb13u4; gcc 16.2.0), built with `gcc -Wall -Wextra -pthread`. <details><summary>Raw output</summary> macOS: ``` cancel while disabled: FAIL - pending cancel ignored, thread still blocked in read() after 3 s (x3) cancel while disabled + pthread_testcancel(): PASS - thread cancelled at the cancellation point (x3) cancel while enabled (control): PASS - thread cancelled at the cancellation point (x3) cancel while enabled, before read(): PASS - thread cancelled at the cancellation point (x3) ``` Linux: ``` cancel while disabled: PASS - thread cancelled at the cancellation point (x3) cancel while disabled + pthread_testcancel(): PASS - thread cancelled at the cancellation point (x3) cancel while enabled (control): PASS - thread cancelled at the cancellation point (x3) cancel while enabled, before read(): PASS - thread cancelled at the cancellation point (x3) ``` </details> The first row is the bug. The `gap` row shows that a request made between `pthread_testcancel()` and the syscall is not lost either, so the patch closes the window and does not only make it smaller. glibc acts on the pending request, so on Linux the extra call only checks a flag. `make fate-source` passes. I ran `udp_close_e2e.c` only on macOS. I did not run it on Linux, FreeBSD, NetBSD or OpenBSD. >From 4bd18f4640bfadbb775076ce1ea7e0846a2fb4ea Mon Sep 17 00:00:00 2001 From: Derek Cyrus-Chow <[email protected]> Date: Thu, 1 Oct 2026 14:49:31 +0100 Subject: [PATCH] avformat/udp: act on a pending cancel request before recvfrom() udp_close() stops the circular buffer RX thread with pthread_cancel() and then waits in pthread_join(). The RX thread keeps cancellation disabled while it handles each packet and enables it only around recvfrom(), relying on recvfrom() being a cancellation point. On Darwin, a pthread_cancel() that arrives while the target has cancellation disabled only marks the request as pending in user space. Neither re-enabling cancellation nor any later recvfrom() acts on it, so the request is lost and the thread never exits: it keeps reading while packets arrive, and blocks in recvfrom() once they stop. udp_close() then waits in pthread_join() forever. The request is lost only if it arrives while the thread handles a packet, so closing an input that is receiving data often hangs, while closing an idle input does not. Call pthread_testcancel() after cancellation is enabled again. It acts on a request that arrived while cancellation was disabled; a request that arrives later reaches the thread while cancellation is enabled and interrupts recvfrom() as before. Win32 does not use pthread_cancel() here (and w32pthreads has no pthread_testcancel()), so skip it there. Tested on macOS 26 (arm64) with a loop that opens a udp:// input, reads while a local sender floods the port, and closes it: without this patch every run of 200 cycles hung in udp_close() within the first 10 cycles (5/5 runs), with the RX thread still looping over recvfrom(); with it, 10/10 runs completed. Signed-off-by: Derek Cyrus-Chow <[email protected]> --- libavformat/udp.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/libavformat/udp.c b/libavformat/udp.c index 0874d18397..90d62f6acb 100644 --- a/libavformat/udp.c +++ b/libavformat/udp.c @@ -550,6 +550,12 @@ static void *circular_buffer_task_rx( void *_URLContext) see "General Information" / "Thread Cancellation Overview" in Single Unix. */ pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, &old_cancelstate); +#ifndef _WIN32 + /* A cancel request that arrived while cancellation was disabled is + only pending; Darwin does not act on it in recvfrom(), so the + thread would block and udp_close() would hang in pthread_join(). */ + pthread_testcancel(); +#endif pkt_header.pkt_size = recvfrom(s->udp_fd, s->tmp + sizeof(pkt_header), sizeof(s->tmp) - sizeof(pkt_header), 0, (struct sockaddr *)&pkt_header.addr, &pkt_header.addr_len); pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, &old_cancelstate); pthread_mutex_lock(&s->mutex); -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
