CV-Bowen opened a new pull request, #21971:
URL: https://github.com/apache/nuttx/pull/21971

   ## Summary
   
   rpmsg_virtio receives with its own kthread per device: the rx interrupt
   moves every non-RT buffer to a readyrx list and wakes the thread, which
   then runs the endpoint callbacks one by one. A callback that blocks (for
   example in `rpmsg_wait()` for a tx buffer) stalls the whole device, and
   other transports cannot reuse the scheme.
   
   This PR moves rx scheduling into the rpmsg framework:
   
   - `drivers/rpmsg: dispatch rx in per-device work queues`: each device
     creates `CONFIG_RPMSG_WQUEUE_NUMBER` work queues; the rx interrupt
     queues a buffer with `rpmsg_queue_rx_work()` on the queue picked by the
     endpoint priority (bound RT endpoints are still handled in the
     interrupt). `rpmsg_wait()` called from one of these queues runs the
     queue's pending works while it waits, bounded by a recursion limit, so
     a blocking callback no longer stalls rx. The kthread and the
     readyrx/freerx lists in rpmsg_virtio are removed.
   - `drivers/rpmsg: add an option to share the work queues between
     devices`: `CONFIG_RPMSG_WQUEUE_GLOBAL` creates one set of queues in
     `rpmsg_initialize()` for all devices.
   - `drivers/rpmsg: fix rpmsg_wait() not returning in the recursive path`:
     at the recursion limit `rpmsg_wait()` returned an error although the
     semaphore could still be posted; keep waiting instead. Adds
     `rpmsg_tickwait()` with a timeout (`rpmsg_wait()` becomes a wrapper)
     and passes the remaining time to the transport wait op.
   
   ## Impact
   
   - Kconfig: `RPMSG_VIRTIO_PRIORITY` / `RPMSG_VIRTIO_STACKSIZE` are replaced
     by `RPMSG_WQUEUE_PRIORITY` / `RPMSG_WQUEUE_STACKSIZE` (same defaults),
     plus `RPMSG_WQUEUE_NUMBER` (default 1) and `RPMSG_WQUEUE_GLOBAL`
     (default n). No in-tree defconfig sets the old options. `RPMSG` now
     selects `SCHED_WORKQUEUE`.
   - The rx thread is named `rpmsg-<cpu>-<n>` instead of `rpmsg-virtio`.
   - Public API: new `rpmsg_tickwait()`; `rpmsg_wait()` keeps its signature
     (now a macro), so the existing callers are unchanged.
   - Private API in `drivers/rpmsg/rpmsg.h`: the transport `wait`/`post` ops
     lose the semaphore argument, and `rpmsg_register()` takes the number of
     rx works to preallocate; rpmsg_port and rpmsg_router_edge pass 0.
   
   ## Testing
   
   Host: Ubuntu 22.04.5 LTS x86_64, QEMU version 10.2.1, sim: cc (Ubuntu 
13.1.0-8ubuntu1~22.04) 13.1.0, qemu-armv8a: aarch64-none-elf-gcc (Arm GNU 
Toolchain 12.2.Rel1 (Build arm-12.24)) 12.2.1 20221205
   
   qemu-armv8a:rpserver + qemu-armv8a:rpproxy (rptun ivshmem; the pair also
   talks over rpmsg_port_uart). A test-only patch (not in this PR) logs, once
   per endpoint, whether the ping callback runs in the interrupt and in
   which task. `rpmsg ping all 20 1024 3 0` pings with the normal endpoint,
   `... 11 0` with the RT one:
   
   | | master | this PR, `RPMSG_WQUEUE_NUMBER=1` | `=3` |
   |---|---|---|---|
   | normal endpoint, virtio | task `rpmsg-virtio` | task `rpmsg-<cpu>-0` | 
task `rpmsg-<cpu>-1` (default priority maps to the middle queue) |
   | RT endpoint, virtio | in irq | in irq | in irq |
   | RT endpoint, port_uart | task `rpmsg-uart-rx` | same | same |
   | 20 x 1024 pings, both sides, both endpoints | pass | pass | pass |
   
   With `=3` each device has three queues at priority 224..226 (`ps`), and
   with `RPMSG_WQUEUE_GLOBAL=y`, `RPMSG_WQUEUE_NUMBER=2` there is one shared
   pair `rpmsg-glb-0/1`; pings and rpmsgfs (`cat /sp/version` from the
   proxy) pass there too.
   
   <details><summary>ping path, this PR, RPMSG_WQUEUE_NUMBER=3</summary>
   
   ```
   [proxy] PINGPATH rpmsg-ping in irq=0 task=rpmsg-server-1
   [proxy] PINGPATH rpmsg-ping-rt in irq=0 task=rpmsg-uart-rx
   [proxy] PINGPATH rpmsg-ping-rt in irq=1 task=-
   [server] PINGPATH rpmsg-ping in irq=0 task=rpmsg-proxy-1
   [server] PINGPATH rpmsg-ping-rt in irq=0 task=rpmsg-uart-rx
   [server] PINGPATH rpmsg-ping-rt in irq=1 task=-
   ```
   </details>
   
   <details><summary>ping path, master</summary>
   
   ```
   [proxy] PINGPATH rpmsg-ping in irq=0 task=rpmsg-virtio
   [proxy] PINGPATH rpmsg-ping-rt in irq=0 task=rpmsg-uart-rx
   [proxy] PINGPATH rpmsg-ping-rt in irq=1 task=-
   [server] PINGPATH rpmsg-ping in irq=0 task=rpmsg-virtio
   [server] PINGPATH rpmsg-ping-rt in irq=0 task=rpmsg-uart-rx
   [server] PINGPATH rpmsg-ping-rt in irq=1 task=-
   ```
   </details>
   
   <details><summary>ps, RPMSG_WQUEUE_NUMBER=3 / RPMSG_WQUEUE_GLOBAL=y</summary>
   
   ```
     TID   PID  PPID PRI POLICY   TYPE    NPX STATE    EVENT     SIGMASK        
    STACK    USED FILLED COMMAND
       9     0     0 224 FIFO     Kthread   - Waiting  Semaphore 
0000000000000000 0008064 0001280  15.8%  rpmsg-proxy-0
      10     0     0 225 FIFO     Kthread   - Waiting  Semaphore 
0000000000000000 0008064 0002224  27.5%  rpmsg-proxy-1
      11     0     0 226 FIFO     Kthread   - Waiting  Semaphore 
0000000000000000 0008064 0002128  26.3%  rpmsg-proxy-2
       1     0     0 224 FIFO     Kthread   - Waiting  Semaphore 
0000000000000000 0008064 0001280  15.8%  rpmsg-glb-0
       2     0     0 225 FIFO     Kthread   - Waiting  Semaphore 
0000000000000000 0008064 0002224  27.5%  rpmsg-glb-1
   ```
   </details>
   
   Every commit builds with no warning on sim:rpserver,
   qemu-armv8a:rpproxy, and sim:rpserver with `RPMSG_WQUEUE_GLOBAL=y`.
   
   No regression in rpmsg: sim:rpserver/rpproxy and 
qemu-armv8a:rpserver/rpproxy boot and pass the fixed rpmsg cases (dump, ping, 
rpmsgfs; 15/15). qemu-armv8a was built with LIBC_ENVPATH and BINFMT_CHECKEXEC 
disabled, which current master needs for nxinit to start nsh there; unrelated 
to this change.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to