Public bug reported: With 0.7.4+ds-1 from 19.10, usbguard may stop responding to events when recvmsg fails with ENOBUFS. To reproduce:
while /bin/true ; do sudo udevadm control --reload-rules sudo udevadm trigger sudo udevadm settle --timeout=3 done Eventually, this pops out in the journald logs: usbguard-daemon[12488]: ueventProcessRead: failed to read pending uevent: rc=-1 errno=105 usbguard-daemon[12488]: UEventDeviceManager thread: UEvent device manager: recvmsg: No buffer space available and usbguard will no longer process events until restarted (sudo systemctl restart usbguard). I then installed usbguard 0.7.5 from the desktop team's PPA[1], ran the loop and saw the equivalent error pop out: usbguard-daemon[5958]: [1575487887.438] (E) ueventProcessRead: failed to read pending uevent: rc=-1 errno=105 usbguard-daemon[5958]: [1575487887.438] (E) UEventDeviceManager thread: UEvent device manager: recvmsg: No buffer space available This is coming from https://github.com/USBGuard/usbguard/blob/master/src/Library/UEventDeviceManager.cpp#L528 where recvmsg() is returning -1 with 'No buffer space available' (ENOBUFS). Looking at sysdeps/gnu/errlist.c in glibc, ENOBUFS is documented as: #ifdef ENOBUFS /* TRANS The kernel's buffers for I/O operations are all in use. In GNU, this TRANS error is always synonymous with @code{ENOMEM}; you may get one or the TRANS other from network operations. */ [ERR_REMAP (ENOBUFS)] = N_("No buffer space available"), It seems that usbguard should be able to handle transient ENOBUFS scenarios with uevent storms and recover when the kernel's buffers are back in order. In my testing, I treated ENOBUFS similarly to EAGAIN/EWOULDBLOCK by logging a warning and simply returning[2]: usbguard-daemon[2405]: ueventProcessRead: failed to read pending uevent (returning): rc=-1 errno=105 It appears usbguard continues to function (eg, if I do the loop for a long time, usbguard remains responsive and processes other uevents, but I'm not sure this is the correct approach since, for example, UEventDeviceManager::ueventOpen() speaks to this error condition by saying: "Set a 1MiB receive buffer on the netlink socket to avoid ENOBUFS error in recvmsg"). [1]https://launchpad.net/~ubuntu-desktop/+archive/ubuntu/usbguard [2]exploratory patch (do not commit in Ubuntu without upstream confirmation) Index: usbguard-0.7.4+ds/src/Library/UEventDeviceManager.cpp =================================================================== --- usbguard-0.7.4+ds.orig/src/Library/UEventDeviceManager.cpp +++ usbguard-0.7.4+ds/src/Library/UEventDeviceManager.cpp @@ -468,6 +468,12 @@ namespace usbguard << "reading from uevent source would block thread execution"; return; } + else if (saved_errno == ENOBUFS) { + USBGUARD_LOG(Error) << "ueventProcessRead: " + << "failed to read pending uevent (returning): " + << "rc=" << rc << " errno=" << saved_errno; + return; + } else { USBGUARD_LOG(Error) << "ueventProcessRead: " << "failed to read pending uevent: " ** Affects: usbguard Importance: Unknown Status: Unknown ** Affects: usbguard (Ubuntu) Importance: Undecided Status: New ** Description changed: With 0.7.4+ds-1 from 19.10, usbguard may stop responding to events when recvmsg fails with ENOBUFS. To reproduce: - while /bin/true ; do - sudo udevadm control --reload-rules - sudo udevadm trigger - sudo udevadm settle --timeout=3 - done + while /bin/true ; do + sudo udevadm control --reload-rules + sudo udevadm trigger + sudo udevadm settle --timeout=3 + done Eventually, this pops out in the journald logs: - usbguard-daemon[12488]: ueventProcessRead: failed to read pending uevent: rc=-1 errno=105 - usbguard-daemon[12488]: UEventDeviceManager thread: UEvent device manager: recvmsg: No buffer space available + usbguard-daemon[12488]: ueventProcessRead: failed to read pending uevent: rc=-1 errno=105 + usbguard-daemon[12488]: UEventDeviceManager thread: UEvent device manager: recvmsg: No buffer space available and usbguard will no longer process events until restarted (sudo systemctl restart usbguard). I then installed usbguard 0.7.5 from the desktop team's PPA[1], ran the loop and saw the equivalent error pop out: - usbguard-daemon[5958]: [1575487887.438] (E) ueventProcessRead: failed to read pending uevent: rc=-1 errno=105 - usbguard-daemon[5958]: [1575487887.438] (E) UEventDeviceManager thread: UEvent device manager: recvmsg: No buffer space available + usbguard-daemon[5958]: [1575487887.438] (E) ueventProcessRead: failed to read pending uevent: rc=-1 errno=105 + usbguard-daemon[5958]: [1575487887.438] (E) UEventDeviceManager thread: UEvent device manager: recvmsg: No buffer space available This is coming from https://github.com/USBGuard/usbguard/blob/master/src/Library/UEventDeviceManager.cpp#L528 where recvmsg() is returning -1 with 'No buffer space available' (ENOBUFS). Looking at sysdeps/gnu/errlist.c in glibc, ENOBUFS is documented as: - #ifdef ENOBUFS - /* - TRANS The kernel's buffers for I/O operations are all in use. In GNU, this - TRANS error is always synonymous with @code{ENOMEM}; you may get one or the - TRANS other from network operations. */ - [ERR_REMAP (ENOBUFS)] = N_("No buffer space available"), + #ifdef ENOBUFS + /* + TRANS The kernel's buffers for I/O operations are all in use. In GNU, this + TRANS error is always synonymous with @code{ENOMEM}; you may get one or the + TRANS other from network operations. */ + [ERR_REMAP (ENOBUFS)] = N_("No buffer space available"), It seems that usbguard should be able to handle transient ENOBUFS scenarios with uevent storms and recover when the kernel's buffers are back in order. In my testing, I treated ENOBUFS similarly to EAGAIN/EWOULDBLOCK by logging a warning and simply returning[2]: - usbguard-daemon[2405]: ueventProcessRead: failed to read pending + usbguard-daemon[2405]: ueventProcessRead: failed to read pending uevent (returning): rc=-1 errno=105 It appears usbguard continues to function (eg, if I do the loop for a long time, usbguard remains responsive and processes other uevents, but I'm not sure this is the correct approach since, for example, UEventDeviceManager::ueventOpen() speaks to this error condition by saying: "Set a 1MiB receive buffer on the netlink socket to avoid - ENOBUFS error in recvmsg." - + ENOBUFS error in recvmsg"). [1]https://launchpad.net/~ubuntu-desktop/+archive/ubuntu/usbguard [2]exploratory patch (do not commit in Ubuntu without upstream confirmation) Index: usbguard-0.7.4+ds/src/Library/UEventDeviceManager.cpp =================================================================== --- usbguard-0.7.4+ds.orig/src/Library/UEventDeviceManager.cpp +++ usbguard-0.7.4+ds/src/Library/UEventDeviceManager.cpp @@ -468,6 +468,12 @@ namespace usbguard - << "reading from uevent source would block thread execution"; - return; - } + << "reading from uevent source would block thread execution"; + return; + } + else if (saved_errno == ENOBUFS) { + USBGUARD_LOG(Error) << "ueventProcessRead: " + << "failed to read pending uevent (returning): " + << "rc=" << rc << " errno=" << saved_errno; + return; + } - else { - USBGUARD_LOG(Error) << "ueventProcessRead: " - << "failed to read pending uevent: " + else { + USBGUARD_LOG(Error) << "ueventProcessRead: " + << "failed to read pending uevent: " -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1855189 Title: usbguard stops responding when recvmsg receives ENOBUFS To manage notifications about this bug go to: https://bugs.launchpad.net/usbguard/+bug/1855189/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
