wyr-7 opened a new pull request, #18339:
URL: https://github.com/apache/nuttx/pull/18339
The rpmsg_get_tx_buffer_size() and rpmsg_get_rx_buffer_size() may return
negative error codes (e.g., -EPERM which is -2003). When storing the MIN()
result in an unsigned size_t variable, negative values overflow to large
positive values, causing incorrect buffer size calculations.
## Summary
Fix integer overflow bug in rpmsgfs_readdir_handler() when rpmsg buffer size
functions return error codes. The `rpmsg_get_tx_buffer_size()` and
`rpmsg_get_rx_buffer_size()` APIs return ssize_t and may return negative error
codes like -EPERM (-2003) when the endpoint is not ready. Previously, the
result was stored in an unsigned `size_t` variable, causing negative values to
overflow into very large positive numbers. This led to incorrect buffer size
calculations and potential memory issues.
This fix:
Change the size variable type from size_t to ssize_t to properly handle
negative return values.
Add explicit error checking for negative buffer sizes before use.
Add boundary check to ensure size >= len before string operations.
## Impact
Impact on security : fixes potential buffer overflow due to incorrect size
calculation
## Testing
Testing log in QEMU ARMV8:
qemu-system-aarch64 -cpu cortex-a53 -nographic \
wyr@hp
20:13:29
-machine virt,virtualization=on,gic-version=3 \
-chardev stdio,id=con,mux=on -serial chardev:con \
-object
memory-backend-file,discard-data=on,id=shmmem-shmem0,mem-path=/dev/shm/my_shmem0,size=4194304,share=yes
\
-device ivshmem-plain,id=shmem0,memdev=shmmem-shmem0,addr=0xb \
-device virtio-serial-device,bus=virtio-mmio-bus.0 \
-chardev socket,path=/tmp/rpmsg_port_uart_socket,server=on,wait=off,id=foo \
-device virtconsole,chardev=foo \
-mon chardev=con,mode=readline -kernel ./nuttx/cmake_out/v8a_server/nuttx \
-gdb tcp::7775
[ 0.000000] [ 0] [ INFO] [server] pci_register_rptun_ivshmem_driver:
Register ivshmem driver, id=0, cpuname=proxy, master=1
[ 0.000000] [ 3] [ INFO] [server] pci_scan_bus: pci_scan_bus for bus 0
[ 0.000000] [ 3] [ INFO] [server] pci_scan_bus: class = 00000600,
hdr_type = 00000000
[ 0.000000] [ 3] [ INFO] [server] pci_scan_bus: 00:00 [1b36:0008]
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar0 set bad mask
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar1 set bad mask
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar2 set bad mask
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar3 set bad mask
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar4 set bad mask
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar5 set bad mask
[ 0.000000] [ 3] [ INFO] [server] pci_scan_bus: class = 00000200,
hdr_type = 00000000
[ 0.000000] [ 3] [ INFO] [server] pci_scan_bus: 00:08 [1af4:1000]
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar0:
mask64=fffffffe 32bytes
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar1:
mask64=fffffff0 4096bytes
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar2 set bad mask
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar3 set bad mask
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar4:
mask64=fffffffffffffff0 16384bytes
[ 0.000000] [ 3] [ INFO] [server] pci_scan_bus: class = 00000500,
hdr_type = 00000000
[ 0.000000] [ 3] [ INFO] [server] pci_scan_bus: 00:58 [1af4:1110]
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar0:
mask64=fffffff0 256bytes
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar1 set bad mask
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar2:
mask64=fffffffffffffff0 4194304bytes
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar4 set bad mask
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar5 set bad mask
[ 0.000000] [ 3] [ INFO] [server] ivshmem_probe: shmem addr=0x8000400000
size=4194304 reg=0x10001000
[ 0.000000] [ 3] [ INFO] [server] rptun_ivshmem_probe: shmem
addr=0x8000400000 size=4194304
NuttShell (NSH)
server>
server>
server>
server> [ 0.000000] [ 0] [ INFO] [proxy]
pci_register_rptun_ivshmem_driver: Register ivshmem driver, id=0,
cpuname=server, master=0
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: pci_scan_bus for bus 0
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: class = 00000600,
hdr_type = 00000000
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: 00:00 [1b36:0008]
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar0 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar1 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar2 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar3 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar4 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar5 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: class = 00000200,
hdr_type = 00000000
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: 00:08 [1af4:1000]
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar0:
mask64=fffffffe 32bytes
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar1:
mask64=fffffff0 4096bytes
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar2 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar3 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar4:
mask64=fffffffffffffff0 16384bytes
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: class = 00000500,
hdr_type = 00000000
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: 00:58 [1af4:1110]
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar0:
mask64=fffffff0 256bytes
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar1 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar2:
mask64=fffffffffffffff0 4194304bytes
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar4 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar5 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] ivshmem_probe: shmem addr=0x8000400000
size=4194304 reg=0x10001000
[ 0.000000] [ 3] [ INFO] [proxy] rptun_ivshmem_probe: shmem
addr=0x8000400000 size=4194304
[ 0.000000] [ 3] [ INFO] [proxy] rptun_ivshmem_probe: Start the wdog
server> ls /dev/rpmsg
/dev/rpmsg:
proxy
proxy2
server> cat /proc/version
NuttX version 0.0.0 8c5f0e5ba43-dirty Feb 3 2026 20:12:48
qemu-armv8a/rpserver
server>
qemu-system-aarch64 -cpu cortex-a53 -nographic \
wyr@hp
20:13:27
-machine virt,virtualization=on,gic-version=3 \
-chardev stdio,id=con,mux=on -serial chardev:con \
-object
memory-backend-file,discard-data=on,id=shmmem-shmem0,mem-path=/dev/shm/my_shmem0,size=4194304,share=yes
\
-device ivshmem-plain,id=shmem0,memdev=shmmem-shmem0,addr=0xb \
-device virtio-serial-device,bus=virtio-mmio-bus.0 \
-chardev socket,path=/tmp/rpmsg_port_uart_socket,server=off,id=foo \
-device virtconsole,chardev=foo \
-mon chardev=con,mode=readline -kernel ./nuttx/cmake_out/v8a_proxy/nuttx \
-gdb tcp::7776
- Ready to Boot Primary CPU
- Boot from EL2
- Boot from EL1
- Boot to C runtime for OS Initialize
[ 0.000000] [ 0] [ INFO] [proxy] pci_register_rptun_ivshmem_driver:
Register ivshmem driver, id=0, cpuname=server, master=0
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: pci_scan_bus for bus 0
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: class = 00000600,
hdr_type = 00000000
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: 00:00 [1b36:0008]
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar0 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar1 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar2 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar3 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar4 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar5 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: class = 00000200,
hdr_type = 00000000
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: 00:08 [1af4:1000]
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar0:
mask64=fffffffe 32bytes
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar1:
mask64=fffffff0 4096bytes
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar2 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar3 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar4:
mask64=fffffffffffffff0 16384bytes
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: class = 00000500,
hdr_type = 00000000
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: 00:58 [1af4:1110]
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar0:
mask64=fffffff0 256bytes
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar1 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar2:
mask64=fffffffffffffff0 4194304bytes
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar4 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar5 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] ivshmem_probe: shmem addr=0x8000400000
size=4194304 reg=0x10001000
[ 0.000000] [ 3] [ INFO] [proxy] rptun_ivshmem_probe: shmem
addr=0x8000400000 size=4194304
[ 0.000000] [ 3] [ INFO] [proxy] rptun_ivshmem_probe: Start the wdog
NuttShell (NSH)
proxy>
proxy> ls /dev/rpmsg
/dev/rpmsg:
server
server2
proxy>
proxy> mount -t rpmsgfs -o cpu=server,fs=/proc /proc_server
proxy> cat /proc_server/version
NuttX version 0.0.0 8c5f0e5ba43-dirty Feb 3 2026 20:12:48
qemu-armv8a/rpserver
proxy>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
