CV-Bowen opened a new pull request, #21832:
URL: https://github.com/apache/nuttx/pull/21832
## Summary
Fixes and small features for epoll and the poll core.
- `fs/epoll: support polling the epoll fd`: `epoll_do_poll()` returned OK
without registering the waiter, so an epoll fd watched by `poll()` or by
another epoll never woke up (libuv `uv_run_embed()` does this).
- `fs/epoll: push the TLS cleanup before file_get()`: a thread killed
outside the semaphore wait leaked the epoll file reference.
- `fs/epoll: use a recursive lock for the epoll head`: a thread killed
while holding `eph->lock` deadlocked in `epoll_do_close()` run by its
TLS cleanup.
- `fs/epoll: add edge-triggered (EPOLLET) support`: `EPOLLET` was accepted
but behaved as level-triggered.
- `fs/epoll: fix ET starvation and lost ET events`: two ET bugs, a node
stranded after an empty notification, and an event lost when
`maxevents` is smaller than the ready set.
- `fs/poll: make POLLALWAYS a two-sided handshake`: epoll sets
`POLLALWAYS` on every node, so every driver notification, even one the
user did not ask for, made `epoll_wait()` tear the fd down and set it up
again while reporting nothing. A driver now has to set `POLLALWAYS` in
the notified eventset to ask for that callback.
- `fs/eventfd`, `fs/mqueue: skip poll setup notification when no event is
requested`: both notified `POLLOUT` on every setup, which an `EPOLLIN`
watcher can never use.
The commits come from different authors and touch the same functions of
`fs_epoll.c`, so they are kept together in order.
## Impact
- `EPOLLET` now has edge-triggered semantics; it was silently
level-triggered before.
- An epoll fd can be watched by `poll()`/`select()`/epoll.
`CONFIG_EPOLL_NPOLLWAITERS` (default 2) sets how many waiters it takes.
- `POLLALWAYS` (kernel-only, `include/nuttx/poll.h`): a callback for a
notification with no requested event now needs `POLLALWAYS` in both the
waiter's `events` and the driver's notified eventset. No in-tree driver
sets it today, so epoll no longer re-arms on such notifications.
Out-of-tree drivers that rely on the re-arm must pass `POLLALWAYS` to
`poll_notify()`. `POLLALWAYS` is also no longer copied into `revents`.
- No API/ABI change for applications.
## Testing
Host: Ubuntu 22.04.5 LTS x86_64, QEMU version 10.2.1, sim: cc (Ubuntu
13.1.0-8ubuntu1~22.04) 13.1.0, qemu-armv8a: aarch64-none-elf-gcc (Arm GNU
Toolchain 12.2.Rel1 (Build arm-12.24)) 12.2.1 20221205
**Targeted test.** `sim:nsh` with a test program built as `hello`
(source below), plus:
```
CONFIG_EVENT_FD=y
CONFIG_EVENT_FD_POLL=y
CONFIG_PIPES=y
CONFIG_CANCELLATION_POINTS=y
```
```
cmake -B build -DBOARD_CONFIG=sim:nsh -GNinja # then add the options above
cmake --build build -t olddefconfig && cmake --build build
./build/nuttx
nsh> hello
```
| case | master | this PR |
|------|--------|---------|
| `poll()` on an epoll fd, member pipe written after 100 ms | ret=0, timeout
| ret=1, POLLIN |
| LT pipe, data left unread: 2 waits | 1, 1 | 1, 1 |
| ET pipe: wait, wait, write, wait | 1, 1, 1 | 1, 0, 1 |
| 2 ET pipes ready, `maxevents=1`: 2 waits return distinct fds | no (same fd
twice) | yes |
| async cancel of a thread blocked in `epoll_wait()`, then `close(epfd)`, 20
rounds | done | done |
| 32 idle eventfd + 32 idle mqueue watched for `EPOLLIN`, 2000 x
`epoll_wait(0)` | 25397 ns/call | 561 ns/call |
The last row was also measured at intermediate commits: 38294 ns/call
after `fix ET starvation`, 559 ns/call after `POLLALWAYS` handshake, and 561
ns/call at the
head. The handshake removes the empty callbacks; the eventfd/mqueue
commits remove the `poll_notify()` call itself.
<details><summary>this PR</summary>
```
nsh> hello
nested: poll(epfd) ret=1 revents=1
lt: wait1=1 wait2=1 (expect 1 1)
et: wait1=1 wait2=0 wait3=1 (expect 1 0 1)
et_full: wait1=1 wait2=1 distinct=1 (expect 1 1 1)
bench: 32 idle eventfd + 32 idle mqueue, 2000 x epoll_wait(0): 1123 us, 561
ns/call
cancel: 20 rounds done
p86 epolltest: PASS (0 failures)
```
</details>
<details><summary>master</summary>
```
nsh> hello
nested: poll(epfd) ret=0 revents=0
FAIL line 71: ret == 1 && (pfd.revents & POLLIN) != 0
lt: wait1=1 wait2=1 (expect 1 1)
et: wait1=1 wait2=1 wait3=1 (expect 1 0 1)
FAIL line 110: n1 == 1 && n2 == 0 && n3 == 1 && ev[0].data.fd == et[0]
et_full: wait1=1 wait2=1 distinct=0 (expect 1 1 1)
FAIL line 144: n1 == 1 && n2 == 1 && ev.data.fd != fd1
bench: 32 idle eventfd + 32 idle mqueue, 2000 x epoll_wait(0): 50794 us,
25397 ns/call
cancel: 20 rounds done
p86 epolltest: FAIL (3 failures)
```
</details>
<details><summary>test program</summary>
```c
/* Test-only: exercise the epoll changes of P86. Replaces hello_main.c in
* the P86 apps worktree; never part of a PR.
*/
#include <nuttx/config.h>
#include <sys/epoll.h>
#include <sys/eventfd.h>
#include <sys/time.h>
#include <mqueue.h>
#include <fcntl.h>
#include <poll.h>
#include <pthread.h>
#include <stdio.h>
#include <string.h>
#include <time.h>
#include <unistd.h>
#define NIDLE 32
#define NLOOPS 2000
static int g_fail;
#define CHECK(c) do { if (!(c)) { printf("FAIL line %d: %s\n", \
__LINE__, #c); g_fail++; } } while (0)
static unsigned long now_us(void)
{
struct timespec ts;
clock_gettime(CLOCK_MONOTONIC, &ts);
return ts.tv_sec * 1000000ul + ts.tv_nsec / 1000;
}
static int add(int epfd, int fd, uint32_t events)
{
struct epoll_event ev;
memset(&ev, 0, sizeof(ev));
ev.events = events;
ev.data.fd = fd;
return epoll_ctl(epfd, EPOLL_CTL_ADD, fd, &ev);
}
/* 1. poll() on an epoll fd wakes when a member becomes ready. */
static void *writer(void *arg)
{
usleep(100 * 1000);
write((int)(intptr_t)arg, "x", 1);
return NULL;
}
static void test_nested(void)
{
struct pollfd pfd;
pthread_t t;
int p[2];
int epfd;
int ret;
epfd = epoll_create1(0);
pipe(p);
CHECK(add(epfd, p[0], EPOLLIN) == 0);
pfd.fd = epfd;
pfd.events = POLLIN;
pthread_create(&t, NULL, writer, (void *)(intptr_t)p[1]);
ret = poll(&pfd, 1, 1000);
pthread_join(t, NULL);
printf("nested: poll(epfd) ret=%d revents=%x\n", ret, pfd.revents);
CHECK(ret == 1 && (pfd.revents & POLLIN) != 0);
close(p[0]);
close(p[1]);
close(epfd);
}
/* 2. LT reports again while data is unread; ET reports once per write. */
static void test_lt_et(void)
{
struct epoll_event ev[2];
int lt[2];
int et[2];
int epfd;
int n1;
int n2;
int n3;
epfd = epoll_create1(0);
pipe(lt);
pipe(et);
CHECK(add(epfd, lt[0], EPOLLIN) == 0);
CHECK(add(epfd, et[0], EPOLLIN | EPOLLET) == 0);
write(lt[1], "x", 1);
n1 = epoll_wait(epfd, ev, 2, 100);
n2 = epoll_wait(epfd, ev, 2, 100);
printf("lt: wait1=%d wait2=%d (expect 1 1)\n", n1, n2);
CHECK(n1 == 1 && n2 == 1);
read(lt[0], ev, 1);
CHECK(epoll_wait(epfd, ev, 2, 50) == 0);
write(et[1], "x", 1);
n1 = epoll_wait(epfd, ev, 2, 100);
n2 = epoll_wait(epfd, ev, 2, 100);
write(et[1], "y", 1);
n3 = epoll_wait(epfd, ev, 2, 100);
printf("et: wait1=%d wait2=%d wait3=%d (expect 1 0 1)\n", n1, n2, n3);
CHECK(n1 == 1 && n2 == 0 && n3 == 1 && ev[0].data.fd == et[0]);
close(lt[0]);
close(lt[1]);
close(et[0]);
close(et[1]);
close(epfd);
}
/* 3. ET with maxevents smaller than the ready set loses no event. */
static void test_et_full(void)
{
struct epoll_event ev;
int a[2];
int b[2];
int epfd;
int n1;
int n2;
int fd1;
epfd = epoll_create1(0);
pipe(a);
pipe(b);
CHECK(add(epfd, a[0], EPOLLIN | EPOLLET) == 0);
CHECK(add(epfd, b[0], EPOLLIN | EPOLLET) == 0);
write(a[1], "x", 1);
write(b[1], "x", 1);
n1 = epoll_wait(epfd, &ev, 1, 100);
fd1 = ev.data.fd;
n2 = epoll_wait(epfd, &ev, 1, 100);
printf("et_full: wait1=%d wait2=%d distinct=%d (expect 1 1 1)\n",
n1, n2, n2 == 1 && ev.data.fd != fd1);
CHECK(n1 == 1 && n2 == 1 && ev.data.fd != fd1);
close(a[0]);
close(a[1]);
close(b[0]);
close(b[1]);
close(epfd);
}
/* 4. Idle eventfd/mqueue members watched for EPOLLIN: cost per wait. */
static void bench_idle(void)
{
struct epoll_event ev;
struct mq_attr attr;
mqd_t mq[NIDLE];
int efd[NIDLE];
unsigned long t;
int epfd;
int i;
memset(&attr, 0, sizeof(attr));
attr.mq_maxmsg = 4;
attr.mq_msgsize = 8;
epfd = epoll_create1(0);
for (i = 0; i < NIDLE; i++)
{
char name[16];
efd[i] = eventfd(0, 0);
CHECK(add(epfd, efd[i], EPOLLIN) == 0);
snprintf(name, sizeof(name), "/p86mq%d", i);
mq[i] = mq_open(name, O_RDWR | O_CREAT, 0666, &attr);
CHECK(add(epfd, (int)mq[i], EPOLLIN) == 0);
}
CHECK(epoll_wait(epfd, &ev, 1, 0) == 0);
t = now_us();
for (i = 0; i < NLOOPS; i++)
{
epoll_wait(epfd, &ev, 1, 0);
}
t = now_us() - t;
printf("bench: %d idle eventfd + %d idle mqueue, %d x epoll_wait(0): "
"%lu us, %lu ns/call\n", NIDLE, NIDLE, NLOOPS, t,
t * 1000 / NLOOPS);
for (i = 0; i < NIDLE; i++)
{
char name[16];
close(efd[i]);
mq_close(mq[i]);
snprintf(name, sizeof(name), "/p86mq%d", i);
mq_unlink(name);
}
close(epfd);
}
/* 5. Cancel a thread blocked in epoll_wait(), then close: no hang. */
static void *waiter(void *arg)
{
struct epoll_event ev;
/* epoll_wait() is not a cancellation point; cancel asynchronously so
* the thread dies inside it and the TLS cleanup runs.
*/
pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL);
epoll_wait((int)(intptr_t)arg, &ev, 1, -1);
return NULL;
}
static void test_cancel(void)
{
pthread_t t;
void *r;
int p[2];
int epfd;
int i;
for (i = 0; i < 20; i++)
{
epfd = epoll_create1(0);
pipe(p);
add(epfd, p[0], EPOLLIN);
pthread_create(&t, NULL, waiter, (void *)(intptr_t)epfd);
usleep(10 * 1000);
pthread_cancel(t);
pthread_join(t, &r);
CHECK(r == PTHREAD_CANCELED);
close(p[0]);
close(p[1]);
CHECK(close(epfd) == 0);
}
printf("cancel: 20 rounds done\n");
}
int main(int argc, FAR char *argv[])
{
test_nested();
test_lt_et();
test_et_full();
bench_idle();
test_cancel();
printf("p86 epolltest: %s (%d failures)\n", g_fail ? "FAIL" : "PASS",
g_fail);
return g_fail;
}
```
</details>
The cancel case does not reproduce the reference leak or the deadlock on
sim; it only checks that cancel and close still work with the changes.
**apps/testing/fs/epoll** (`cmocka_epoll`, `sim:nsh` +
`ALLOW_MIT_COMPONENTS LIBC_REGEX TESTING_CMOCKA TESTING_EPOLL TIMER_FD
NET_LOCAL NET_LOCAL_STREAM EVENT_FD`): epoll01 and epoll02 pass,
epoll03 fails at `connect()` of a non-blocking local stream socket
(`ret == -1 && errno == EINPROGRESS`, epoll.c:400) on both master and
this PR.
**Regression.** sim:rpserver/rpproxy and qemu-armv8a:rpserver/rpproxy
boot and pass the fixed rpmsg cases (dump, ping, rpmsgfs; 15/15).
qemu-armv8a was built with LIBC_ENVPATH and BINFMT_CHECKEXEC disabled,
which current master needs for nxinit to start nsh there; unrelated to
this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]