sdelavega opened a new issue, #22135:
URL: https://github.com/apache/nuttx/issues/22135
**Environment:** NuttX master 295bf8c385 and 13.1.0 · Arch: xtensa
(ESP32-S3), also seen on the simulator; the code is architecture independent ·
Area: POSIX / OS components · Host: Ubuntu 24.04 (WSL2)
### Description / Steps to reproduce the issue
`pthread_cond_wait()` and `pthread_cond_clockwait()` can lose a wakeup on
SMP. A thread that starts waiting *after* `pthread_cond_broadcast()` or
`pthread_cond_signal()` can take the post that was meant for a thread that was
already waiting. The thread that was already waiting then blocks with nothing
left to wake it. POSIX requires broadcast to unblock every thread blocked on
the condition when it is called, and signal at least one of them.
Commit ef20f417f0 ("libc/pthread: Make cond wait release and block atomic",
PR #21834) fixed this on a single CPU. Its message says: "This does not close
the window against threads on another CPU under SMP." This issue covers that
SMP case. I did not find an existing issue for it.
#### Mechanism (master 295bf8c385)
`libs/libc/pthread/pthread_condwait.c`:
```c
96 sched_lock();
97
98 atomic_add(COND_WAIT_COUNT(cond), 1);
99 ret = pthread_mutex_breaklock(mutex, &nlocks);
100
101 status = -nxsem_wait_uninterruptible(&cond->sem);
102 sched_unlock();
```
`libs/libc/pthread/pthread_condbroadcast.c` (`pthread_condsignal.c` lines
78-81 is the same with one post):
```c
81 if (atomic_cmpxchg(COND_WAIT_COUNT(cond), &wcnt, wcnt - 1))
...
87 ret = -nxsem_post(&cond->sem);
```
`pthread_cond_clockwait()` has the same sequence. The post goes to the
shared `cond->sem` and is not bound to a thread. Between line 99 (mutex
released) and line 101 (blocked), the waiter is counted but not yet on the
semaphore's wait list. `sched_lock()` keeps other threads on the same CPU out
of that window, but not threads on another CPU:
```
CPU0: consumer C CPU1: producer P
lock(m); buffer empty
wait_count++ (1); release m
... still before nxsem_wait ... lock(m); put item
broadcast: wait_count 1 -> 0, post
(nobody blocked: semcount 0 -> 1)
put more items until the buffer is full
wait: wait_count++ (1); release m
nxsem_wait: takes C's post, returns
buffer still full: wait again (blocks)
nxsem_wait: semcount 0 -> -1, blocks
```
Both threads now sleep, with `wait_count` 2 and `semcount` -2, and the
consumer sleeps with a full buffer. Nothing will post again. This matches the
state in the ef20f417f0 report (`wait_count == 2`, `semcount == -2`), which was
hit on one CPU before that commit.
#### Reproducer
`testing/sched/condsteal` (nuttx-apps patch at the end of this issue; I can
open a PR for it): one mutex, one condition, a bounded buffer, producers and
consumers that both wait on that condition in predicate loops and wake the
other side after every item. The main thread is a watchdog: if no item moves
for 3 s it prints what it sees and exits with status 1.
```
condsteal [-n items] [-r rounds] [-p producers] [-c consumers] [-b slots]
[-t stall_ms] [-T timeout_us] [-s]
```
A failing run looks like this:
```
nsh> condsteal -n 100000
condsteal: 1 round(s) of 100000 items, 1 producer(s), 1 consumer(s), 4
slots, broadcast, timeout 0 us
condsteal: STALL in round 1 after 3000 ms: produced 172 consumed 168 of
100000, count 4/4, waiting producers 1 consumers 1, waits 86
condsteal: lost wakeup: a consumer waits with 4 item(s) in the buffer
condsteal: FAILED
```
To run it:
- ESP32-S3 on Espressif QEMU (`esp-develop-9.2.2-20260417`):
`./tools/configure.sh esp32s3-devkit:qemu_debug`, enable
`CONFIG_TESTING_CONDSTEAL`, build, then `qemu-system-xtensa -nographic -machine
esp32s3 -drive file=nuttx.merged.bin,if=mtd,format=raw` and run `condsteal -n
100000` at the NSH prompt. My build host has no `genromfs`, so I also set
`CONFIG_ETC_ROMFS=n` and `CONFIG_INIT_ENTRYPOINT="nsh_main"`, and I disabled
networking, Wi-Fi and hostfs. With Espressif QEMU I use
`CONFIG_ESP32S3_RX_FIFO_THRD=1`, because QEMU sometimes does not raise the UART
RX timeout interrupt and NSH then never sees a short command. For UP, also set
`CONFIG_SMP=n`.
- sim: `./tools/configure.sh sim:smp`, enable `CONFIG_TESTING_CONDSTEAL`,
run `./nuttx`, then `condsteal`.
#### Observed results
ESP32-S3 QEMU, `esp32s3-devkit:qemu_debug`, 2 CPUs for SMP. Each launch is
one `condsteal -n 100000` (100,000 items) with a 3 s stall watchdog. A cell
gives the launches that stalled, and in brackets the median number of items
consumed before the stall.
| Build | broadcast, 1P/1C | signal (`-s`), 1P/1C | broadcast, 1 slot (`-b
1`) | broadcast, 2P/2C | timed (`-T 1000`, 1P/2C) |
|---|---|---|---|---|---|
| 13.1.0 (2b5c0de5e9) SMP | 100/100 (98) | 100/100 (106) | 100/100 (24) |
0/100 | 0/50 |
| 13.1.0 UP | 2/100 (67,458) | 5/100 (26,484) | 10/100 (32,719) | 0/100 |
0/50 |
| master (295bf8c385) SMP | **100/100 (170)** | **100/100 (260)** |
**100/100 (36)** | 0/100 | 0/50 |
| master UP | 0/100 | 0/100 | 0/100 | 0/100 | 0/50 |
| master + proposed fix, SMP | 0/100 | 0/100 | 0/100 | 0/100 | 0/50 |
| master + proposed fix, UP | 0/100 | 0/100 | 0/100 | 0/100 | 0/50 |
- The 2-producer/2-consumer variant does not reproduce the bug, even on
master SMP: the other threads keep posting. With one producer and one consumer,
every launch on SMP stalls.
- The timed variant cannot stall, because a timeout makes the waiter check
again. It exercises timeouts racing wakeups and checks that
`pthread_cond_destroy()` returns 0 after each round.
- sim:smp (4 CPUs) on master: 18 of 20 launches of `condsteal -n 5000`
stalled. With the fix: 0 of 20 stalled; 19 passed, and in 1 the sim process
itself died (see Testing done).
#### On real silicon (ESP32-S3, 2 CPUs)
A dual-core ESP32-S3 (QFN56 rev v0.2, 16 MB flash), `esp32s3-devkit:nsh`
with `CONFIG_SMP=y`, 2 CPUs (`ps` shows both idle threads), console on
USB-Serial/JTAG. Images and logs are available on request.
| Build | broadcast, 1P/1C | signal (`-s`) | broadcast, 1 slot (`-b 1`) |
2P/2C | timed (`-T 1000`, 1P/2C) | ostest |
|---|---|---|---|---|---|---|
| master (295bf8c385) | passed 20 rounds, then **stalled in round 1** of a
200-round run | **stalled in round 2**, and in round 1 on a rerun | **stalled
in round 2** | not run | not run | status 0 (1 run) |
| master + proposed fix | 200 rounds passed | 100 rounds passed | 100 rounds
passed | 5 rounds passed | 5 rounds passed (21 timeouts) | status 0 in 2 of 3
runs; 1 run failed `mutex_test: ERROR: The pthread is still running!` |
Every stall printed the lost-wakeup diagnosis (a consumer waiting with items
in the buffer, or a producer waiting with free slots). The one ostest failure
with the fix is in `mutex_test` (timed mutex lock, no condition variable); it
did not recur in two further runs, and I have only one master run to compare,
so I can't yet say whether it is pre-existing.
#### Proposed fix
Patch: at the end of this issue (against 295bf8c385). It is a proposal for
discussion, not yet ready to merge: see the known limits below. I can open a PR
once the design is agreed.
Each waiter queues a node on its own stack, holding its own semaphore, in
priority order on a list in `pthread_cond_t`. It does this under a new lock in
`pthread_cond_t`, before it releases the mutex. Under that lock,
`pthread_cond_signal()` unlinks and posts the first node, and
`pthread_cond_broadcast()` all of them; each posts exactly the nodes it
unlinked. A post can only be taken by the thread it was sent to, so the
waiter's window between releasing the mutex and blocking no longer matters, on
any CPU, and the `sched_lock()` is no longer needed.
- Timeout: the waiter takes the lock and unlinks its node. If a signal has
already unlinked it, the post was made under the lock and is already there. The
waiter takes it, so that no post is left on its stack, and returns 0 rather
than ETIMEDOUT, so that the signal is not lost.
- Cancellation (the wait is a cancellation point): handled like a timeout,
except that a waiter that was already signalled passes the signal on with
`pthread_cond_signal()`. POSIX requires that a canceled waiter not consume a
signal when other threads are blocked.
- `pthread_cond_destroy()` returns EBUSY exactly while a thread is queued.
Today a canceled waiter is never removed from `wait_count`, so destroy returns
EBUSY until a later signal or broadcast posts for it.
- ABI: `pthread_cond_t` becomes `{mutex_t lock; clockid_t clockid; FAR
struct pthread_cond_waiter_s *waiters;}` instead of `{sem_t sem; clockid_t
clockid; int wait_count;}`. The size is the same on 32-bit targets without
`CONFIG_LIBC_MUTEX_BACKTRACE`. It grows on 64-bit targets (a pointer instead of
an `int`) and by the backtrace array when `CONFIG_LIBC_MUTEX_BACKTRACE > 0`.
`PTHREAD_COND_INITIALIZER` changes accordingly.
- Everything stays in libc. The code uses only nxmutex, nxsem and
`sched_getparam()`, so it calls no kernel-only interface.
Alternatives I considered:
- A gate or generation scheme that keeps the single shared semaphore (as in
pthreads-win32 or old glibc). It needs a second semaphore plus counters, and
new waiters must block until every thread woken by the last broadcast has run.
That is a priority inversion on the caller's mutex.
- A waiter that finds it took a post it was not entitled to and re-posts it.
With several such waiters this cannot be decided from counters alone, and it
busy-waits.
Known limits of the proposal (for discussion):
- **Asynchronous termination of a waiter.** The nodes are on the waiters'
stacks. If a waiter is terminated without running its own exit path, its node
stays on the list and points into its freed stack. The next signal or broadcast
then posts into that memory, and `pthread_cond_destroy()` stays EBUSY until
then. With `CONFIG_CANCELLATION_POINTS=y`, `pthread_cancel()`, the forced
cancellation on group exit and `task_delete()` wake a blocked waiter with
ECANCELED, and it unlinks itself. But with `CONFIG_CANCELLATION_POINTS=n`,
which is the Kconfig default, all three call `nxtask_terminate()` on the
blocked waiter. So does an asynchronous-type cancel of a waiter that has not
blocked yet. ostest's cancel test does exactly this in that configuration: test
1a cancels a thread blocked in `pthread_cond_wait()`, and test 3 then
broadcasts the same condition. It passed on my builds only because the freed
stack was not yet reused. Today these cases only leave `wait_count` too high.
The fix I
would propose before merging is a hook in the task recovery path
(`nxtask_recover()`), like `pthread_mutex_inconsistent()` for robust mutexes:
the waiter records its node in its TLS, and the hook unlinks it from the
condition variable. Signal and broadcast post under the lock, so a signaller
that is terminated part-way leaves the not-yet-posted nodes on the list instead
of stranding them.
- **Process-shared condition variables across address spaces
(`CONFIG_BUILD_KERNEL`) are not supported by this design.** The waiter nodes
are on the waiters' stacks, which another process cannot reach. With the old
kernel-managed semaphore this case likely worked, so this is a regression for
it. A cheap guard would be for `pthread_cond_init()` to return ENOTSUP for a
`PTHREAD_PROCESS_SHARED` attribute under `CONFIG_BUILD_KERNEL`, so that it
fails loudly instead of corrupting memory. I would add that guard if the
per-waiter design is kept, but it does not remove the regression.
- The queue order uses the waiter's priority when it starts waiting. A
priority change while it waits does not reorder the list, which the semaphore
wait list does.
#### Testing done
- Every SMP and UP column in the table above, with and without the fix. With
the fix that is 450 launches and 45 million items per build without a stall.
- `ostest` from NSH on ESP32-S3 QEMU, master and master + fix, SMP and UP, 2
runs each: all ended with `Exiting with status 0`. The condition variable,
cancel and timed wait tests ran in each. For these runs I commented out
`memmem_test()` and `strto_test()` in `ostest_main.c`: unmodified, ostest stops
with an `ASSERT` in "libc tests" on this board, on master and with the fix
alike. 13.1.0's ostest, which has no libc tests, passed on SMP and UP.
- sim:ostest (UP, `CONFIG_TESTING_OSTEST_LOOPS=5`): within a 600 s limit,
master and the fix both completed 2 full loops with the same output and no
errors, then both stalled in the third loop's wdog test.
- sim:smp under WSL2: long condsteal runs make the sim process crash
(SIGSEGV) or hang, on master as well as with the fix. On master this also
happens with `-T 2000`, where timeouts prevent the lost-wakeup stall. So I did
not use sim:smp for stress.
- All QEMU builds had `CONFIG_CANCELLATION_POINTS=n` (the `qemu_debug`
default), so the deferred-cancellation path of the fix was not exercised there.
- `tools/nxstyle` is clean on the changed files and on the test.
- Real hardware: see "On real silicon" above.
- Not tested: protected or kernel builds.
### Host information
Build and test host: Ubuntu 24.04 under WSL2 (x86_64). Toolchain
crosstool-NG `esp-14.2.0_20241119` (GCC 14.2.0). Espressif QEMU
`esp-develop-9.2.2-20260417`.
### Patches
<details><summary>Proposed fix:
<code>0001-libc-pthread-Wake-condition-variable-waiters-through.patch</code>
(nuttx, against 295bf8c385)</summary>
```diff
From fee67ad868f44f8ace8ba225e1c53f1d192eccf4 Mon Sep 17 00:00:00 2001
From: Stephen de la Vega <[email protected]>
Date: Fri, 20 Nov 2026 14:07:33 -0500
Subject: [PATCH] libc/pthread: Wake condition variable waiters through
per-waiter nodes.
pthread_cond_wait() counts itself in wait_count, releases the mutex and
then blocks on the shared cond->sem. pthread_cond_signal() and
pthread_cond_broadcast() post that semaphore once per counted waiter.
A post is not bound to a thread: if it is made while a counted waiter
is still between releasing the mutex and blocking, any thread that
reaches nxsem_wait() first takes it, including one that started waiting
after the post. The waiter the post was meant for then blocks with
nothing left to wake it. With producers and consumers on one condition
this deadlocks: the stray wakeup goes to a thread whose predicate is
still false, it waits again, and the thread whose predicate is true
sleeps.
Commit ef20f417f0 ("libc/pthread: Make cond wait release and block
atomic") closes the window on one CPU with sched_lock(), but a thread
on another CPU can still signal and then wait in it. On ESP32-S3 QEMU
with SMP (2 CPUs) and on sim:smp, the testing/sched/condsteal test (one
mutex, one condition, a bounded buffer, broadcast after every item)
stalls this way within the first few thousand items.
Give every waiter a node on its stack with its own semaphore. The node
is queued in priority order under a new lock in pthread_cond_t before
the mutex is released. Under that lock, signal unlinks and posts the
first node and broadcast all of them. A post can only be taken by the
thread it was sent to, so it does not matter how long that thread
takes to block, on any CPU, and the sched_lock() window is no longer
needed.
A waiter that times out or is canceled takes the lock and unlinks its
node. If a signal or broadcast has already unlinked it, the post was
made under the lock and is already there: the waiter takes it, so that
the semaphore is not left posted on its stack. Then a timed wait
returns 0 (the signal is reported, not lost), and a canceled wait
passes the signal on with pthread_cond_signal(), as POSIX requires.
Before, a timed-out waiter could decrement wait_count after a signal
had already counted it, leaving wait_count negative and a post that
the next waiter took as a spurious wakeup, and a canceled waiter was
never removed from wait_count, so pthread_cond_destroy() returned EBUSY
until a later signal or broadcast posted for it.
pthread_cond_destroy() now returns EBUSY exactly while a thread is
queued.
pthread_cond_t changes from {sem_t sem; clockid_t clockid; int
wait_count} to {mutex_t lock; clockid_t clockid; FAR struct
pthread_cond_waiter_s *waiters}. The size is the same on 32-bit
targets without CONFIG_LIBC_MUTEX_BACKTRACE; it grows on 64-bit targets
(pointer instead of int) and by the backtrace array when
CONFIG_LIBC_MUTEX_BACKTRACE > 0. PTHREAD_COND_INITIALIZER changes
accordingly, so code that uses it must be rebuilt. Everything stays in
libc and uses only nxmutex, nxsem and sched_getparam().
Known limits:
- A waiter that is terminated asynchronously while it waits leaves its
node on the list, pointing into its freed stack, and the next signal
or broadcast posts into that memory. Deferred cancellation, the
default with CONFIG_CANCELLATION_POINTS=y, wakes the waiter with
ECANCELED and it unlinks itself. But with
CONFIG_CANCELLATION_POINTS=n (the Kconfig default), pthread_cancel()
of a waiter, the forced cancellation on group exit, and
task_delete() of a waiting task go through nxtask_terminate() without
running the waiter, and so does an asynchronous cancellation of a
waiter that is not blocked. Before this change the same cases only
left wait_count too high. A fix needs a hook in the task recovery
path, like pthread_mutex_inconsistent() for robust mutexes.
- Process-shared condition variables used from different address
spaces (CONFIG_BUILD_KERNEL) are not supported: the waiter nodes are
on the waiters' stacks. The shared kernel-managed semaphore made
that case work before, so this is a regression for it.
Signed-off-by: Stephen de la Vega <[email protected]>
Assisted-by: Claude Code
---
include/pthread.h | 9 +-
libs/libc/pthread/pthread.h | 24 ++-
libs/libc/pthread/pthread_condbroadcast.c | 48 +++--
libs/libc/pthread/pthread_condclockwait.c | 50 +----
libs/libc/pthread/pthread_conddestroy.c | 22 +--
libs/libc/pthread/pthread_condinit.c | 18 +-
libs/libc/pthread/pthread_condsignal.c | 30 ++-
libs/libc/pthread/pthread_condwait.c | 225 ++++++++++++++++++----
8 files changed, 286 insertions(+), 140 deletions(-)
diff --git a/include/pthread.h b/include/pthread.h
index dca4df1008..ac5886a3d4 100644
--- a/include/pthread.h
+++ b/include/pthread.h
@@ -247,11 +247,14 @@ typedef struct pthread_condattr_s pthread_condattr_t;
# define __PTHREAD_CONDATTR_T_DEFINED 1
#endif
+struct pthread_cond_waiter_s; /* Private to libc */
+
struct pthread_cond_s
{
- sem_t sem;
+ mutex_t lock; /* Protects waiters */
clockid_t clockid;
- int wait_count;
+ FAR struct pthread_cond_waiter_s *waiters; /* Blocked threads, highest
+ * priority first */
};
#ifndef __PTHREAD_COND_T_DEFINED
@@ -259,7 +262,7 @@ typedef struct pthread_cond_s pthread_cond_t;
# define __PTHREAD_COND_T_DEFINED 1
#endif
-#define PTHREAD_COND_INITIALIZER {SEM_INITIALIZER(0), CLOCK_REALTIME, 0}
+#define PTHREAD_COND_INITIALIZER {NXMUTEX_INITIALIZER, CLOCK_REALTIME, NULL}
struct pthread_mutexattr_s
{
diff --git a/libs/libc/pthread/pthread.h b/libs/libc/pthread/pthread.h
index e4a2e6bd4c..3dc9230470 100644
--- a/libs/libc/pthread/pthread.h
+++ b/libs/libc/pthread/pthread.h
@@ -73,8 +73,6 @@
# define mutex_setprioceiling(m,p,o) nxmutex_setprioceiling(m,p,o)
#endif
-#define COND_WAIT_COUNT(cond) ((FAR atomic_t *)&(cond)->wait_count)
-
/* Validate if a pthread_attr_t object is valid.
*
* This macro checks:
@@ -142,10 +140,32 @@
((mutex)->mutex.sem.flags & SEM_TYPE_MUTEX))
#endif
+/****************************************************************************
+ * Public Types
+
****************************************************************************/
+
+/* A thread blocked in pthread_cond_wait() or pthread_cond_clockwait(). The
+ * node lives on the stack of the waiting thread and is linked into
+ * pthread_cond_t::waiters while the thread waits.
+ */
+
+struct pthread_cond_waiter_s
+{
+ FAR struct pthread_cond_waiter_s *flink; /* Next waiter */
+ sem_t sem; /* Posted once to wake it */
+ int priority; /* Priority when it queued */
+ bool signalled; /* Taken off the list by a
+ * signal or broadcast */
+};
+
/****************************************************************************
* Public Function Prototypes
****************************************************************************/
+int pthread_cond_dowait(FAR pthread_cond_t *cond,
+ FAR pthread_mutex_t *mutex, clockid_t clockid,
+ FAR const struct timespec *abstime);
+
#ifndef CONFIG_PTHREAD_MUTEX_UNSAFE
int pthread_mutex_take(FAR struct pthread_mutex_s *mutex,
FAR const struct timespec *abs_timeout);
diff --git a/libs/libc/pthread/pthread_condbroadcast.c
b/libs/libc/pthread/pthread_condbroadcast.c
index 792ef1c422..b59aee95fe 100644
--- a/libs/libc/pthread/pthread_condbroadcast.c
+++ b/libs/libc/pthread/pthread_condbroadcast.c
@@ -31,7 +31,8 @@
#include <errno.h>
#include <nuttx/debug.h>
-#include <nuttx/atomic.h>
+#include <nuttx/mutex.h>
+#include <nuttx/semaphore.h>
#include "pthread.h"
@@ -72,25 +73,42 @@ int pthread_cond_broadcast(FAR pthread_cond_t *cond)
}
else
{
- int wcnt = atomic_read(COND_WAIT_COUNT(cond));
-
- /* Loop until all of the waiting threads have been restarted. */
-
- while (wcnt > 0)
+ FAR struct pthread_cond_waiter_s *waiter;
+ FAR struct pthread_cond_waiter_s *next;
+ int status;
+
+ /* Take every waiter off the list and post it, in priority order.
+ * Threads that start waiting after this point queue on the now empty
+ * list and are not woken. The posts are made under the lock, so a
+ * waiter that finds itself signalled under the lock knows that its
+ * post is already there. A node lives on the stack of its waiter
+ * until the waiter takes the post, so read the link first.
+ */
+
+ nxmutex_lock(&cond->lock);
+ waiter = cond->waiters;
+ cond->waiters = NULL;
+
+ while (waiter != NULL)
{
- if (atomic_cmpxchg(COND_WAIT_COUNT(cond), &wcnt, wcnt - 1))
- {
- /* Post the condition semaphore to wake up a waiting thread.
- * Only the highest priority waiting thread will get to
execute
- */
-
- ret = -nxsem_post(&cond->sem);
+ next = waiter->flink;
+ waiter->signalled = true;
- /* Decrement the waiter count */
+ /* Each node's semaphore starts at 0 and is posted only here, so
+ * the post cannot fail; keep the first error and go on anyway,
+ * so that the other waiters are still woken.
+ */
- wcnt--;
+ status = -nxsem_post(&waiter->sem);
+ if (ret == OK)
+ {
+ ret = status;
}
+
+ waiter = next;
}
+
+ nxmutex_unlock(&cond->lock);
}
sinfo("Returning %d\n", ret);
diff --git a/libs/libc/pthread/pthread_condclockwait.c
b/libs/libc/pthread/pthread_condclockwait.c
index 23687307b7..5be99f84a6 100644
--- a/libs/libc/pthread/pthread_condclockwait.c
+++ b/libs/libc/pthread/pthread_condclockwait.c
@@ -36,14 +36,7 @@
#include <assert.h>
#include <nuttx/debug.h>
-#include <nuttx/atomic.h>
-#include <nuttx/irq.h>
-#include <nuttx/wdog.h>
-#include <nuttx/signal.h>
#include <nuttx/cancelpt.h>
-#include <nuttx/pthread.h>
-#include <nuttx/atomic.h>
-#include <nuttx/sched.h>
#include "pthread.h"
@@ -77,7 +70,6 @@ int pthread_cond_clockwait(FAR pthread_cond_t *cond,
FAR const struct timespec *abstime)
{
int ret = OK;
- int status;
sinfo("cond=%p mutex=%p abstime=%p\n", cond, mutex, abstime);
@@ -121,47 +113,7 @@ int pthread_cond_clockwait(FAR pthread_cond_t *cond,
else
{
- unsigned int nlocks;
-
- sinfo("Give up mutex...\n");
-
- sched_lock();
-
- atomic_add(COND_WAIT_COUNT(cond), 1);
-
- /* Give up the mutex */
-
- ret = pthread_mutex_breaklock(mutex, &nlocks);
- if (ret == 0)
- {
- status = nxsem_clockwait_uninterruptible(&cond->sem,
- clockid, abstime);
- if (status < 0)
- {
- ret = -status;
- }
- }
-
- sched_unlock();
-
- /* Reacquire the mutex (retaining the ret). */
-
- sinfo("Re-locking...\n");
-
- status = pthread_mutex_restorelock(mutex, nlocks);
-
- /* When the wait times out (ETIMEDOUT), we must decrement
- * wait_count to maintain proper accounting.
- */
-
- if (ret == ETIMEDOUT)
- {
- atomic_sub(COND_WAIT_COUNT(cond), 1);
- }
- else if (ret == 0)
- {
- ret = status;
- }
+ ret = pthread_cond_dowait(cond, mutex, clockid, abstime);
}
leave_cancellation_point();
diff --git a/libs/libc/pthread/pthread_conddestroy.c
b/libs/libc/pthread/pthread_conddestroy.c
index 8fc30ce6d9..6de189c21f 100644
--- a/libs/libc/pthread/pthread_conddestroy.c
+++ b/libs/libc/pthread/pthread_conddestroy.c
@@ -30,7 +30,7 @@
#include <nuttx/debug.h>
#include <errno.h>
-#include <nuttx/atomic.h>
+#include <nuttx/mutex.h>
#include "pthread.h"
@@ -60,7 +60,6 @@
int pthread_cond_destroy(FAR pthread_cond_t *cond)
{
int ret = OK;
- int sval = 0;
sinfo("cond=%p\n", cond);
@@ -75,24 +74,19 @@ int pthread_cond_destroy(FAR pthread_cond_t *cond)
* the state of the condition variable.
*/
- else if (atomic_read(COND_WAIT_COUNT(cond)) > 0)
- {
- ret = EBUSY;
- }
else
{
- ret = nxsem_get_value(&cond->sem, &sval);
- if (ret < 0)
- {
- ret = -ret;
- }
- else if (sval < 0)
+ nxmutex_lock(&cond->lock);
+ if (cond->waiters != NULL)
{
ret = EBUSY;
}
- else
+
+ nxmutex_unlock(&cond->lock);
+
+ if (ret == OK)
{
- ret = -nxsem_destroy(&cond->sem);
+ ret = -nxmutex_destroy(&cond->lock);
if (ret == OK)
{
cond->clockid = 0;
diff --git a/libs/libc/pthread/pthread_condinit.c
b/libs/libc/pthread/pthread_condinit.c
index c13bf06744..516b41fb69 100644
--- a/libs/libc/pthread/pthread_condinit.c
+++ b/libs/libc/pthread/pthread_condinit.c
@@ -27,7 +27,7 @@
#include <nuttx/config.h>
#include <pthread.h>
-#include <semaphore.h>
+#include <nuttx/mutex.h>
#include <nuttx/debug.h>
#include <errno.h>
@@ -74,18 +74,16 @@ int pthread_cond_init(FAR pthread_cond_t *cond,
ret = EINVAL;
}
- /* Initialize the semaphore contained in the condition structure with
- * initial count = 0
- */
+ /* Initialize the lock that protects the list of waiters */
- else if (sem_init(&cond->sem, 0, 0) != OK)
- {
- ret = get_errno();
- }
else
{
- cond->clockid = attr ? attr->clockid : CLOCK_REALTIME;
- cond->wait_count = 0;
+ ret = -nxmutex_init(&cond->lock);
+ if (ret == OK)
+ {
+ cond->clockid = attr ? attr->clockid : CLOCK_REALTIME;
+ cond->waiters = NULL;
+ }
}
sinfo("Returning %d\n", ret);
diff --git a/libs/libc/pthread/pthread_condsignal.c
b/libs/libc/pthread/pthread_condsignal.c
index 39c07c977a..498431ef67 100644
--- a/libs/libc/pthread/pthread_condsignal.c
+++ b/libs/libc/pthread/pthread_condsignal.c
@@ -30,7 +30,8 @@
#include <errno.h>
#include <nuttx/debug.h>
-#include <nuttx/atomic.h>
+#include <nuttx/mutex.h>
+#include <nuttx/semaphore.h>
#include "pthread.h"
@@ -71,17 +72,28 @@ int pthread_cond_signal(FAR pthread_cond_t *cond)
}
else
{
- int wcnt = atomic_read(COND_WAIT_COUNT(cond));
+ FAR struct pthread_cond_waiter_s *waiter;
- while (wcnt > 0)
+ /* Take the highest priority waiter off the list and post it. Only
+ * that thread can consume the post, even if it has not blocked yet.
+ * The post is made under the lock, so a waiter that finds itself
+ * signalled under the lock knows that its post is already there.
+ * The node lives on the stack of the waiter until the waiter takes
+ * the post, so the post is the last access to it.
+ */
+
+ nxmutex_lock(&cond->lock);
+ waiter = cond->waiters;
+ if (waiter != NULL)
{
- if (atomic_cmpxchg(COND_WAIT_COUNT(cond), &wcnt, wcnt - 1))
- {
- sinfo("Signalling...\n");
- ret = -nxsem_post(&cond->sem);
- break;
- }
+ cond->waiters = waiter->flink;
+ waiter->signalled = true;
+
+ sinfo("Signalling...\n");
+ ret = -nxsem_post(&waiter->sem);
}
+
+ nxmutex_unlock(&cond->lock);
}
sinfo("Returning %d\n", ret);
diff --git a/libs/libc/pthread/pthread_condwait.c
b/libs/libc/pthread/pthread_condwait.c
index be6326c92a..89c29c6d77 100644
--- a/libs/libc/pthread/pthread_condwait.c
+++ b/libs/libc/pthread/pthread_condwait.c
@@ -32,89 +32,191 @@
#include <errno.h>
#include <nuttx/debug.h>
-#include <nuttx/atomic.h>
#include <nuttx/cancelpt.h>
+#include <nuttx/mutex.h>
#include <nuttx/pthread.h>
-#include <nuttx/sched.h>
+#include <nuttx/semaphore.h>
#include "pthread.h"
+/****************************************************************************
+ * Private Functions
+
****************************************************************************/
+
+/****************************************************************************
+ * Name: pthread_cond_unlink
+ *
+ * Description:
+ * Remove a waiter that was not signalled from the list of waiters. The
+ * caller holds cond->lock.
+ *
+
****************************************************************************/
+
+static void pthread_cond_unlink(FAR pthread_cond_t *cond,
+ FAR struct pthread_cond_waiter_s *waiter)
+{
+ FAR struct pthread_cond_waiter_s **link = &cond->waiters;
+
+ while (*link != waiter)
+ {
+ DEBUGASSERT(*link != NULL);
+ link = &(*link)->flink;
+ }
+
+ *link = waiter->flink;
+}
+
/****************************************************************************
* Public Functions
****************************************************************************/
/****************************************************************************
- * Name: int pthread_cond_wait
+ * Name: pthread_cond_dowait
*
* Description:
- * A thread can wait for a condition variable to be signalled or
broadcast.
+ * Common body of pthread_cond_wait() and pthread_cond_clockwait().
+ *
+ * Each waiter queues a node with its own semaphore on its stack, and
+ * does so before it releases the mutex. pthread_cond_signal() and
+ * pthread_cond_broadcast() unlink the nodes they wake and post the
+ * semaphores of exactly those nodes, under cond->lock. A wakeup is thus
+ * bound to a thread that was waiting when it was sent: a thread that
+ * starts waiting later, on this or any other CPU, cannot consume it,
+ * however long the waiter takes between releasing the mutex and
+ * blocking.
*
* Input Parameters:
- * None
+ * cond - The condition variable to wait on
+ * mutex - The mutex that protects the condition, held by the caller
+ * clockid - The clock of abstime
+ * abstime - The absolute timeout, or NULL to wait without a timeout
*
* Returned Value:
- * None
+ * OK (0) on success; a positive errno value on failure. ETIMEDOUT is
+ * returned only if the waiter was not signalled: a signal that races
+ * with the timeout is reported as success.
*
* Assumptions:
+ * The caller has validated cond, mutex and abstime, checked that it
+ * holds the mutex, and entered the cancellation point.
*
****************************************************************************/
-int pthread_cond_wait(FAR pthread_cond_t *cond, FAR pthread_mutex_t *mutex)
+int pthread_cond_dowait(FAR pthread_cond_t *cond,
+ FAR pthread_mutex_t *mutex, clockid_t clockid,
+ FAR const struct timespec *abstime)
{
+ struct pthread_cond_waiter_s waiter;
+ FAR struct pthread_cond_waiter_s **link;
+ struct sched_param param;
+ unsigned int nlocks;
+ bool signalled;
+ bool released;
int status;
int ret;
- sinfo("cond=%p mutex=%p\n", cond, mutex);
+ /* Queue behind the waiters of the same or higher priority, so that
+ * signals keep waking the highest priority waiter first.
+ */
- /* pthread_cond_wait() is a cancellation point */
+ if (sched_getparam(0, ¶m) < 0)
+ {
+ param.sched_priority = SCHED_PRIORITY_MIN;
+ }
- enter_cancellation_point();
+ nxsem_init(&waiter.sem, 0, 0);
+ waiter.priority = param.sched_priority;
+ waiter.signalled = false;
- /* Make sure that non-NULL references were provided. */
+ nxmutex_lock(&cond->lock);
- if (!PTHREAD_COND_IS_VALID(cond) || !PTHREAD_MUTEX_IS_VALID(mutex))
+ link = &cond->waiters;
+ while (*link != NULL && (*link)->priority >= waiter.priority)
{
- ret = EINVAL;
+ link = &(*link)->flink;
}
- /* Make sure that the caller holds the mutex */
+ waiter.flink = *link;
+ *link = &waiter;
- else if (!mutex_is_hold(&mutex->mutex))
- {
- ret = EPERM;
- }
- else
- {
- unsigned int nlocks;
+ nxmutex_unlock(&cond->lock);
- /* Give up the mutex */
+ /* Give up the mutex. From here on, a signal or broadcast that finds the
+ * node posts waiter.sem, whether or not this thread has blocked yet.
+ */
- sinfo("Give up mutex / take cond\n");
+ sinfo("Give up mutex / take cond\n");
- /* Unlock and block atomically, or a later waiter can steal the post
*/
+ ret = pthread_mutex_breaklock(mutex, &nlocks);
+ released = ret == OK;
+ if (released)
+ {
+ if (abstime != NULL)
+ {
+ ret = -nxsem_clockwait_uninterruptible(&waiter.sem, clockid,
+ abstime);
+ }
+ else
+ {
+ ret = -nxsem_wait_uninterruptible(&waiter.sem);
+ }
+ }
- sched_lock();
+ if (ret != OK)
+ {
+ /* Timed out, canceled, or the mutex could not be released. Leave
+ * the list unless a signal or broadcast has already taken the node:
+ * then its post has been made, under cond->lock, and must be
+ * consumed before the node leaves this stack.
+ */
- atomic_add(COND_WAIT_COUNT(cond), 1);
- ret = pthread_mutex_breaklock(mutex, &nlocks);
+ nxmutex_lock(&cond->lock);
+ signalled = waiter.signalled;
+ if (!signalled)
+ {
+ pthread_cond_unlink(cond, &waiter);
+ }
- status = -nxsem_wait_uninterruptible(&cond->sem);
- sched_unlock();
+ nxmutex_unlock(&cond->lock);
- if (ret == OK)
+ if (signalled)
{
- /* Report the first failure that occurs */
+ do
+ {
+ status = nxsem_wait_uninterruptible(&waiter.sem);
+ }
+ while (status < 0);
- ret = status;
+ if (ret == ETIMEDOUT)
+ {
+ /* The signal raced with the timeout: report the wakeup, so
+ * that the signal is not lost.
+ */
+
+ ret = OK;
+ }
+ else
+ {
+ /* A canceled thread must not consume a signal that another
+ * waiter could take: pass it on.
+ */
+
+ pthread_cond_signal(cond);
+ }
}
+ }
- /* Reacquire the mutex.
- *
- * When cancellation points are enabled, we need to hold the mutex
- * when the pthread is canceled and cleanup handlers, if any, are
- * entered.
- */
+ nxsem_destroy(&waiter.sem);
+
+ /* Reacquire the mutex.
+ *
+ * When cancellation points are enabled, we need to hold the mutex
+ * when the pthread is canceled and cleanup handlers, if any, are
+ * entered.
+ */
+ if (released)
+ {
sinfo("Reacquire mutex...\n");
status = pthread_mutex_restorelock(mutex, nlocks);
@@ -126,6 +228,53 @@ int pthread_cond_wait(FAR pthread_cond_t *cond, FAR
pthread_mutex_t *mutex)
}
}
+ return ret;
+}
+
+/****************************************************************************
+ * Name: pthread_cond_wait
+ *
+ * Description:
+ * A thread can wait for a condition variable to be signalled or
broadcast.
+ *
+ * Input Parameters:
+ * None
+ *
+ * Returned Value:
+ * None
+ *
+ * Assumptions:
+ *
+
****************************************************************************/
+
+int pthread_cond_wait(FAR pthread_cond_t *cond, FAR pthread_mutex_t *mutex)
+{
+ int ret;
+
+ sinfo("cond=%p mutex=%p\n", cond, mutex);
+
+ /* pthread_cond_wait() is a cancellation point */
+
+ enter_cancellation_point();
+
+ /* Make sure that non-NULL references were provided. */
+
+ if (!PTHREAD_COND_IS_VALID(cond) || !PTHREAD_MUTEX_IS_VALID(mutex))
+ {
+ ret = EINVAL;
+ }
+
+ /* Make sure that the caller holds the mutex */
+
+ else if (!mutex_is_hold(&mutex->mutex))
+ {
+ ret = EPERM;
+ }
+ else
+ {
+ ret = pthread_cond_dowait(cond, mutex, cond->clockid, NULL);
+ }
+
leave_cancellation_point();
sinfo("Returning %d\n", ret);
return ret;
--
2.43.0
```
</details>
<details><summary>Reproducer: <code>testing/sched/condsteal</code>
(nuttx-apps, against c3b0f42182)</summary>
```diff
From 85c568bd5b3076879fa5d44007a95e71a91bdd6b Mon Sep 17 00:00:00 2001
From: Stephen de la Vega <[email protected]>
Date: Fri, 20 Nov 2026 14:18:44 -0500
Subject: [PATCH] testing/sched/condsteal: Add a condition variable
lost-wakeup
test.
Producers and consumers share one bounded buffer, one mutex and one
condition variable, and wake each other with pthread_cond_broadcast()
(or pthread_cond_signal() with -s) after every item. The main thread
is a watchdog: when no item moves for the stall time (-t, 3 s by
default) it prints what it can see (items in the buffer, how many
producers and consumers are inside pthread_cond_wait()) and exits with
status 1. A consumer that waits with items in the buffer, or a producer
that waits with free slots, means that a wakeup was lost.
-T makes every wait a pthread_cond_timedwait() with a short timeout, so
that timeouts race with wakeups; each round also checks that
pthread_cond_destroy() succeeds afterwards.
Signed-off-by: Stephen de la Vega <[email protected]>
Assisted-by: Claude Code
---
testing/sched/condsteal/CMakeLists.txt | 41 +++
testing/sched/condsteal/Kconfig | 27 ++
testing/sched/condsteal/Make.defs | 25 ++
testing/sched/condsteal/Makefile | 32 ++
testing/sched/condsteal/condsteal_main.c | 432 +++++++++++++++++++++++
5 files changed, 557 insertions(+)
create mode 100644 testing/sched/condsteal/CMakeLists.txt
create mode 100644 testing/sched/condsteal/Kconfig
create mode 100644 testing/sched/condsteal/Make.defs
create mode 100644 testing/sched/condsteal/Makefile
create mode 100644 testing/sched/condsteal/condsteal_main.c
diff --git a/testing/sched/condsteal/CMakeLists.txt
b/testing/sched/condsteal/CMakeLists.txt
new file mode 100644
index 000000000..0a631391e
--- /dev/null
+++ b/testing/sched/condsteal/CMakeLists.txt
@@ -0,0 +1,41 @@
+#
##############################################################################
+# apps/testing/sched/condsteal/CMakeLists.txt
+#
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
+# license agreements. See the NOTICE file distributed with this work for
+# additional information regarding copyright ownership. The ASF licenses
this
+# file to you under the Apache License, Version 2.0 (the "License"); you
may not
+# use this file except in compliance with the License. You may obtain a
copy of
+# the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
under
+# the License.
+#
+#
##############################################################################
+
+if(CONFIG_TESTING_CONDSTEAL)
+
+ set(SRCS condsteal_main.c)
+
+ nuttx_add_application(
+ NAME
+ condsteal
+ PRIORITY
+ ${CONFIG_TESTING_CONDSTEAL_PRIORITY}
+ STACKSIZE
+ ${CONFIG_TESTING_CONDSTEAL_STACKSIZE}
+ MODULE
+ ${CONFIG_TESTING_CONDSTEAL}
+ COMPILE_FLAGS
+ ${FLAGS}
+ SRCS
+ ${SRCS})
+
+endif()
diff --git a/testing/sched/condsteal/Kconfig
b/testing/sched/condsteal/Kconfig
new file mode 100644
index 000000000..4b6c11bfc
--- /dev/null
+++ b/testing/sched/condsteal/Kconfig
@@ -0,0 +1,27 @@
+#
+# For a description of the syntax of this configuration file,
+# see the file kconfig-language.txt in the NuttX tools repository.
+#
+
+config TESTING_CONDSTEAL
+ tristate "Condition variable lost-wakeup test"
+ default n
+ ---help---
+ Producers and consumers share one bounded buffer, one mutex and
+ one condition variable, and wake each other after every item.
+ A watchdog reports a stall, which means that a wakeup was lost,
+ for example when a thread that starts waiting after
+ pthread_cond_broadcast() or pthread_cond_signal() consumes the
+ wakeup of a thread that was already waiting.
+
+if TESTING_CONDSTEAL
+
+config TESTING_CONDSTEAL_PRIORITY
+ int "Task priority"
+ default 100
+
+config TESTING_CONDSTEAL_STACKSIZE
+ int "Stack size"
+ default DEFAULT_TASK_STACKSIZE
+
+endif
diff --git a/testing/sched/condsteal/Make.defs
b/testing/sched/condsteal/Make.defs
new file mode 100644
index 000000000..af2e8ad23
--- /dev/null
+++ b/testing/sched/condsteal/Make.defs
@@ -0,0 +1,25 @@
+############################################################################
+# apps/testing/sched/condsteal/Make.defs
+#
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership. The
+# ASF licenses this file to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance with the
+# License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+############################################################################
+
+ifneq ($(CONFIG_TESTING_CONDSTEAL),)
+CONFIGURED_APPS += $(APPDIR)/testing/sched/condsteal
+endif
diff --git a/testing/sched/condsteal/Makefile
b/testing/sched/condsteal/Makefile
new file mode 100644
index 000000000..fb506786d
--- /dev/null
+++ b/testing/sched/condsteal/Makefile
@@ -0,0 +1,32 @@
+############################################################################
+# apps/testing/sched/condsteal/Makefile
+#
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership. The
+# ASF licenses this file to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance with the
+# License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+############################################################################
+
+include $(APPDIR)/Make.defs
+
+PROGNAME = condsteal
+PRIORITY = $(CONFIG_TESTING_CONDSTEAL_PRIORITY)
+STACKSIZE = $(CONFIG_TESTING_CONDSTEAL_STACKSIZE)
+MODULE = $(CONFIG_TESTING_CONDSTEAL)
+
+MAINSRC = condsteal_main.c
+
+include $(APPDIR)/Application.mk
diff --git a/testing/sched/condsteal/condsteal_main.c
b/testing/sched/condsteal/condsteal_main.c
new file mode 100644
index 000000000..5ae81b197
--- /dev/null
+++ b/testing/sched/condsteal/condsteal_main.c
@@ -0,0 +1,432 @@
+/****************************************************************************
+ * apps/testing/sched/condsteal/condsteal_main.c
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership. The
+ * ASF licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the
+ * License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ *
+
****************************************************************************/
+
+/* Lost-wakeup test for pthread condition variables.
+ *
+ * Producers and consumers share one bounded buffer, one mutex and ONE
+ * condition variable. Each side waits in a predicate loop and wakes the
+ * other side after every change. With a correct condition variable this
+ * cannot deadlock: pthread_cond_broadcast() must unblock every thread that
+ * is blocked on the condition when it is called, and pthread_cond_signal()
+ * at least one of them.
+ *
+ * If a wakeup is lost (for example, when a thread that starts waiting
+ * after the wakeup consumes it in place of a thread that was already
+ * waiting), a consumer can stay blocked with items in the buffer, or a
+ * producer with free space, while the other side also waits. The main
+ * thread is a watchdog: when no item moves for the stall time, it prints
+ * the state it can see and the program exits with status 1.
+ *
+ * Usage: condsteal [-n items] [-r rounds] [-p producers] [-c consumers]
+ * [-b slots] [-t stall_ms] [-T timeout_us] [-s]
+ *
+ * -T wait with pthread_cond_timedwait() and this relative timeout, so
+ * that timeouts race with wakeups. A timeout only makes the thread
+ * check its predicate again.
+ *
+ * -s wake with pthread_cond_signal() instead of
+ * pthread_cond_broadcast(). Only valid with one producer and one
+ * consumer: with more threads of one kind, a signal can legitimately
+ * wake a thread of the same kind and the program would deadlock
+ * even with a correct condition variable.
+ */
+
+/****************************************************************************
+ * Included Files
+
****************************************************************************/
+
+#include <errno.h>
+#include <pthread.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <time.h>
+#include <unistd.h>
+
+/****************************************************************************
+ * Pre-processor Definitions
+
****************************************************************************/
+
+#define CONDSTEAL_MAXTHREADS 8
+#define CONDSTEAL_MAXSLOTS 64
+
+/****************************************************************************
+ * Private Types
+
****************************************************************************/
+
+struct condsteal_s
+{
+ pthread_mutex_t lock;
+ pthread_cond_t cond;
+ unsigned long buffer[CONDSTEAL_MAXSLOTS];
+ unsigned int slots; /* Buffer capacity */
+ unsigned int head; /* Next slot to read */
+ unsigned int count; /* Items in the buffer */
+ unsigned long nitems; /* Items to move in this round */
+ unsigned long produced; /* Items put so far */
+ unsigned long consumed; /* Items taken so far */
+ unsigned long long sum; /* Sum of the consumed values */
+ unsigned int pwaiting; /* Producers inside pthread_cond_wait() */
+ unsigned int cwaiting; /* Consumers inside pthread_cond_wait() */
+ unsigned long waits; /* Calls to pthread_cond_wait() */
+ unsigned long timeouts; /* Waits that returned ETIMEDOUT */
+ unsigned long tmo_us; /* Timed wait in us, or 0: no timeout */
+ bool use_signal; /* Wake with signal, not broadcast */
+};
+
+/****************************************************************************
+ * Private Data
+
****************************************************************************/
+
+static struct condsteal_s g_cs;
+
+/****************************************************************************
+ * Private Functions
+
****************************************************************************/
+
+static void condsteal_wake(FAR struct condsteal_s *cs)
+{
+ int ret;
+
+ if (cs->use_signal)
+ {
+ ret = pthread_cond_signal(&cs->cond);
+ }
+ else
+ {
+ ret = pthread_cond_broadcast(&cs->cond);
+ }
+
+ if (ret != 0)
+ {
+ printf("condsteal: wake failed: %d\n", ret);
+ }
+}
+
+static void condsteal_wait(FAR struct condsteal_s *cs,
+ FAR unsigned int *waiting)
+{
+ int ret;
+
+ (*waiting)++;
+ cs->waits++;
+ if (cs->tmo_us > 0)
+ {
+ struct timespec ts;
+
+ clock_gettime(CLOCK_REALTIME, &ts);
+ ts.tv_nsec += (cs->tmo_us % 1000000) * 1000;
+ ts.tv_sec += cs->tmo_us / 1000000 + ts.tv_nsec / 1000000000;
+ ts.tv_nsec %= 1000000000;
+ ret = pthread_cond_timedwait(&cs->cond, &cs->lock, &ts);
+ if (ret == ETIMEDOUT)
+ {
+ cs->timeouts++;
+ ret = 0;
+ }
+ }
+ else
+ {
+ ret = pthread_cond_wait(&cs->cond, &cs->lock);
+ }
+
+ (*waiting)--;
+
+ if (ret != 0)
+ {
+ printf("condsteal: pthread_cond_wait failed: %d\n", ret);
+ }
+}
+
+static FAR void *condsteal_producer(FAR void *arg)
+{
+ FAR struct condsteal_s *cs = arg;
+
+ pthread_mutex_lock(&cs->lock);
+ while (cs->produced < cs->nitems)
+ {
+ while (cs->count == cs->slots && cs->produced < cs->nitems)
+ {
+ condsteal_wait(cs, &cs->pwaiting);
+ }
+
+ if (cs->produced < cs->nitems)
+ {
+ cs->produced++;
+ cs->buffer[(cs->head + cs->count) % cs->slots] = cs->produced;
+ cs->count++;
+ condsteal_wake(cs);
+ }
+ }
+
+ pthread_mutex_unlock(&cs->lock);
+ return NULL;
+}
+
+static FAR void *condsteal_consumer(FAR void *arg)
+{
+ FAR struct condsteal_s *cs = arg;
+
+ pthread_mutex_lock(&cs->lock);
+ while (cs->consumed < cs->nitems)
+ {
+ while (cs->count == 0 && cs->consumed < cs->nitems)
+ {
+ condsteal_wait(cs, &cs->cwaiting);
+ }
+
+ if (cs->consumed < cs->nitems)
+ {
+ cs->sum += cs->buffer[cs->head];
+ cs->head = (cs->head + 1) % cs->slots;
+ cs->count--;
+ cs->consumed++;
+ condsteal_wake(cs);
+ }
+ }
+
+ /* Let the other consumers see that the work is done */
+
+ pthread_cond_broadcast(&cs->cond);
+ pthread_mutex_unlock(&cs->lock);
+ return NULL;
+}
+
+static unsigned long condsteal_ms(void)
+{
+ struct timespec ts;
+
+ clock_gettime(CLOCK_MONOTONIC, &ts);
+ return (unsigned long)ts.tv_sec * 1000 + ts.tv_nsec / 1000000;
+}
+
+static void condsteal_usage(void)
+{
+ printf("Usage: condsteal [-n items] [-r rounds] [-p producers] "
+ "[-c consumers] [-b slots] [-t stall_ms] [-T timeout_us] "
+ "[-s]\n");
+}
+
+/****************************************************************************
+ * Public Functions
+
****************************************************************************/
+
+int main(int argc, FAR char *argv[])
+{
+ FAR struct condsteal_s *cs = &g_cs;
+ pthread_t threads[2 * CONDSTEAL_MAXTHREADS];
+ unsigned long nitems = 100000;
+ unsigned long stall_ms = 3000;
+ unsigned long timeout_us = 0;
+ unsigned long long expected;
+ unsigned long rounds = 1;
+ unsigned long round;
+ unsigned int nproducers = 1;
+ unsigned int nconsumers = 1;
+ unsigned int slots = 4;
+ unsigned int nthreads;
+ unsigned int i;
+ bool use_signal = false;
+ int opt;
+ int ret;
+
+ while ((opt = getopt(argc, argv, "n:r:p:c:b:t:T:s")) != -1)
+ {
+ switch (opt)
+ {
+ case 'n':
+ nitems = strtoul(optarg, NULL, 0);
+ break;
+
+ case 'r':
+ rounds = strtoul(optarg, NULL, 0);
+ break;
+
+ case 'p':
+ nproducers = strtoul(optarg, NULL, 0);
+ break;
+
+ case 'c':
+ nconsumers = strtoul(optarg, NULL, 0);
+ break;
+
+ case 'b':
+ slots = strtoul(optarg, NULL, 0);
+ break;
+
+ case 't':
+ stall_ms = strtoul(optarg, NULL, 0);
+ break;
+
+ case 'T':
+ timeout_us = strtoul(optarg, NULL, 0);
+ break;
+
+ case 's':
+ use_signal = true;
+ break;
+
+ default:
+ condsteal_usage();
+ return EXIT_FAILURE;
+ }
+ }
+
+ if (nitems == 0 || rounds == 0 || stall_ms == 0 ||
+ nproducers == 0 || nproducers > CONDSTEAL_MAXTHREADS ||
+ nconsumers == 0 || nconsumers > CONDSTEAL_MAXTHREADS ||
+ slots == 0 || slots > CONDSTEAL_MAXSLOTS ||
+ (use_signal && (nproducers != 1 || nconsumers != 1)))
+ {
+ condsteal_usage();
+ return EXIT_FAILURE;
+ }
+
+ expected = (unsigned long long)nitems * (nitems + 1) / 2;
+ printf("condsteal: %lu round(s) of %lu items, %u producer(s), "
+ "%u consumer(s), %u slots, %s, timeout %lu us\n", rounds, nitems,
+ nproducers, nconsumers, slots,
+ use_signal ? "signal" : "broadcast", timeout_us);
+
+ for (round = 1; round <= rounds; round++)
+ {
+ unsigned long last_moved = 0;
+ unsigned long last_ms;
+ bool stalled = false;
+
+ memset(cs, 0, sizeof(*cs));
+ cs->slots = slots;
+ cs->nitems = nitems;
+ cs->use_signal = use_signal;
+ cs->tmo_us = timeout_us;
+ pthread_mutex_init(&cs->lock, NULL);
+ pthread_cond_init(&cs->cond, NULL);
+
+ nthreads = 0;
+ for (i = 0; i < nproducers + nconsumers; i++)
+ {
+ ret = pthread_create(&threads[nthreads], NULL,
+ i < nproducers ? condsteal_producer :
+ condsteal_consumer, cs);
+ if (ret != 0)
+ {
+ printf("condsteal: pthread_create failed: %d\n", ret);
+ return EXIT_FAILURE;
+ }
+
+ nthreads++;
+ }
+
+ /* Watchdog: poll the progress counters */
+
+ last_ms = condsteal_ms();
+ for (; ; )
+ {
+ unsigned long moved;
+ bool done;
+
+ usleep(20000);
+
+ pthread_mutex_lock(&cs->lock);
+ moved = cs->produced + cs->consumed;
+ done = cs->consumed == cs->nitems;
+ if (!done && moved == last_moved &&
+ condsteal_ms() - last_ms >= stall_ms)
+ {
+ stalled = true;
+ printf("condsteal: STALL in round %lu after %lu ms: "
+ "produced %lu consumed %lu of %lu, count %u/%u, "
+ "waiting producers %u consumers %u, waits %lu\n",
+ round, stall_ms, cs->produced, cs->consumed,
+ cs->nitems, cs->count, cs->slots, cs->pwaiting,
+ cs->cwaiting, cs->waits);
+ if (cs->cwaiting > 0 && cs->count > 0)
+ {
+ printf("condsteal: lost wakeup: a consumer waits with "
+ "%u item(s) in the buffer\n", cs->count);
+ }
+
+ if (cs->pwaiting > 0 && cs->count < cs->slots &&
+ cs->produced < cs->nitems)
+ {
+ printf("condsteal: lost wakeup: a producer waits with "
+ "%u free slot(s)\n", cs->slots - cs->count);
+ }
+ }
+
+ pthread_mutex_unlock(&cs->lock);
+
+ if (done || stalled)
+ {
+ break;
+ }
+
+ if (moved != last_moved)
+ {
+ last_moved = moved;
+ last_ms = condsteal_ms();
+ }
+ }
+
+ if (stalled)
+ {
+ /* The blocked threads end when this task exits */
+
+ printf("condsteal: FAILED\n");
+ return EXIT_FAILURE;
+ }
+
+ for (i = 0; i < nthreads; i++)
+ {
+ pthread_join(threads[i], NULL);
+ }
+
+ ret = pthread_cond_destroy(&cs->cond);
+ if (ret != 0)
+ {
+ printf("condsteal: round %lu: pthread_cond_destroy failed: %d\n",
+ round, ret);
+ printf("condsteal: FAILED\n");
+ return EXIT_FAILURE;
+ }
+
+ pthread_mutex_destroy(&cs->lock);
+
+ if (cs->sum != expected)
+ {
+ printf("condsteal: round %lu: sum %llu, expected %llu\n",
+ round, cs->sum, expected);
+ printf("condsteal: FAILED\n");
+ return EXIT_FAILURE;
+ }
+
+ if (rounds > 1 && (round % 10 == 0 || round == rounds))
+ {
+ printf("condsteal: round %lu ok (%lu waits, %lu timeouts)\n",
+ round, cs->waits, cs->timeouts);
+ }
+ }
+
+ printf("condsteal: PASSED\n");
+ return EXIT_SUCCESS;
+}
--
2.43.0
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]