yangsong8-a1 opened a new pull request, #21782:
URL: https://github.com/apache/nuttx/pull/21782
## Summary
When pty_register2() fails to register the slave device, it unregisters the
master with unregister_driver(). That calls pty_unlink(), which calls
pty_destroy() and frees devpair because pp_nopen is 0. The error path then
used devpair again (nxmutex_destroy, nxsem_destroy, kmm_free).
- Return right after unregister_driver() when pseudo-fs operations are
enabled, because pty_unlink() has already freed devpair.
- In pty_destroy(), unregister the slave before releasing the minor
number, so the minor cannot be reused while /dev/pts/N still exists.
- Follow-up: on that path pty_destroy() also freed the minor, and
ptmx_open() freed it a second time (DEBUGASSERT in ptmx_minor_free()).
It also removed the /dev/pts/N node, which may belong to another driver.
Take a reference before unregistering the master, so pty_unlink() keeps
the pair, and free it on the normal error path. The caller frees the
minor once.
## Impact
Only the pty_register2() error path and the order of operations in
pty_destroy(). Normal open and close of ptys does not change.
## Testing
Ubuntu 22.04, gcc 13.1. sim:nsh with CONFIG_PSEUDOTERM (SUSv1) and
CONFIG_MM_KASAN (CONFIG_DEBUG_ASSERTIONS enabled), no new warnings. A test
app opens and closes /dev/ptmx 20 times, with data both ways, then
registers a dummy driver at /dev/pts/0 so that the next /dev/ptmx open
fails in pty_register2():
Before:
```
normal: 20 open/close, fails=0, /dev/pts entries=0
inject: register dummy /dev/pts/0 -> 0
kasan_report: kasan detected a read access error, address at
0x7bec5b52411c,size is 1
```
After:
```
normal: 20 open/close, fails=0, /dev/pts entries=0
inject: register dummy /dev/pts/0 -> 0
inject: open /dev/ptmx -> -1 errno=17 (expect -1 EEXIST=17)
after: 5 open/close, fails=0
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]