Public bug reported: [Impact]
Launchpad's buildd-manager is a complex event-driven daemon that manages several hundred builders on production. As such, it can often end up with more than 1024 file descriptors open. It also uses gpgme from time to time to fetch signing keys for PPAs when dispatching builds that involve PPAs. On focal (and also on jammy), this breaks because `_gpgme_io_select` is implemented using select(2), whose manual page says: "WARNING: select() can monitor only file descriptors numbers that are less than FD_SETSIZE (1024)—an unreasonably low limit for many modern applications—and this limitation will not change. All modern applications should instead use poll(2) or epoll(7), which do not suffer this limitation." This is exactly the limitation we're running into, resulting in mysterious "Too many open files" errors despite resource limits being set to more than comfortable values. Fortunately, this was fixed upstream in https://git.gnupg.org/cgi- bin/gitweb.cgi?p=gpgme.git;a=commitdiff;h=8148237cb4ae20755c06a44d71761c7030973c3d, and lunar has that fix. focal and jammy do not (there may be a similar problem with older LTSes too, but I only care about it back to focal). buildd-manager currently runs on focal. We've been running buildd-manager with a cherry-picked fix from upstream for a few weeks now, and I'd like to get that SRUed back into Ubuntu so that we don't accidentally lose this patch in the event of a security update or similar. [Test Plan] William Grant provided this minimal reproducer: ``` root@tmp:~# python3 Python 3.8.10 (default, May 26 2023, 14:05:08) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import gpg >>> c = gpg.Context() >>> list(c.keylist()) [] >>> fs = [] >>> for i in range(1024): ... fs.append(open('/dev/null')) ... >>> list(c.keylist()) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python3/dist-packages/gpg/core.py", line 751, in keylist key = self.op_keylist_next() File "/usr/lib/python3/dist-packages/gpg/core.py", line 1195, in op_keylist_next raise excp File "/usr/lib/python3/dist-packages/gpg/core.py", line 1190, in op_keylist_next errorcheck(gpgme.gpgme_op_keylist_next(self.wrapped, ptr)) File "/usr/lib/python3/dist-packages/gpg/errors.py", line 129, in errorcheck raise GPGMEError(retval, extradata) gpg.errors.GPGMEError: GPGME: Too many open files ``` [Where problems could occur] `_gpgme_io_select` is pretty central to gpgme, so if something is wrong the package will be pretty broken. On the other hand, that also means any glaring problems are likely to be caught by the package's test suite. ** Affects: gpgme1.0 (Ubuntu) Importance: Undecided Status: Fix Released ** Affects: gpgme1.0 (Ubuntu Focal) Importance: High Assignee: Colin Watson (cjwatson) Status: In Progress ** Affects: gpgme1.0 (Ubuntu Jammy) Importance: High Assignee: Colin Watson (cjwatson) Status: In Progress ** Also affects: gpgme1.0 (Ubuntu Jammy) Importance: Undecided Status: New ** Also affects: gpgme1.0 (Ubuntu Focal) Importance: Undecided Status: New ** Changed in: gpgme1.0 (Ubuntu) Status: New => Fix Released -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to gpgme1.0 in Ubuntu. https://bugs.launchpad.net/bugs/2037595 Title: _gpgme_io_select breaks when more than 1024 file descriptors are open Status in gpgme1.0 package in Ubuntu: Fix Released Status in gpgme1.0 source package in Focal: In Progress Status in gpgme1.0 source package in Jammy: In Progress Bug description: [Impact] Launchpad's buildd-manager is a complex event-driven daemon that manages several hundred builders on production. As such, it can often end up with more than 1024 file descriptors open. It also uses gpgme from time to time to fetch signing keys for PPAs when dispatching builds that involve PPAs. On focal (and also on jammy), this breaks because `_gpgme_io_select` is implemented using select(2), whose manual page says: "WARNING: select() can monitor only file descriptors numbers that are less than FD_SETSIZE (1024)—an unreasonably low limit for many modern applications—and this limitation will not change. All modern applications should instead use poll(2) or epoll(7), which do not suffer this limitation." This is exactly the limitation we're running into, resulting in mysterious "Too many open files" errors despite resource limits being set to more than comfortable values. Fortunately, this was fixed upstream in https://git.gnupg.org/cgi- bin/gitweb.cgi?p=gpgme.git;a=commitdiff;h=8148237cb4ae20755c06a44d71761c7030973c3d, and lunar has that fix. focal and jammy do not (there may be a similar problem with older LTSes too, but I only care about it back to focal). buildd-manager currently runs on focal. We've been running buildd-manager with a cherry-picked fix from upstream for a few weeks now, and I'd like to get that SRUed back into Ubuntu so that we don't accidentally lose this patch in the event of a security update or similar. [Test Plan] William Grant provided this minimal reproducer: ``` root@tmp:~# python3 Python 3.8.10 (default, May 26 2023, 14:05:08) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import gpg >>> c = gpg.Context() >>> list(c.keylist()) [] >>> fs = [] >>> for i in range(1024): ... fs.append(open('/dev/null')) ... >>> list(c.keylist()) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python3/dist-packages/gpg/core.py", line 751, in keylist key = self.op_keylist_next() File "/usr/lib/python3/dist-packages/gpg/core.py", line 1195, in op_keylist_next raise excp File "/usr/lib/python3/dist-packages/gpg/core.py", line 1190, in op_keylist_next errorcheck(gpgme.gpgme_op_keylist_next(self.wrapped, ptr)) File "/usr/lib/python3/dist-packages/gpg/errors.py", line 129, in errorcheck raise GPGMEError(retval, extradata) gpg.errors.GPGMEError: GPGME: Too many open files ``` [Where problems could occur] `_gpgme_io_select` is pretty central to gpgme, so if something is wrong the package will be pretty broken. On the other hand, that also means any glaring problems are likely to be caught by the package's test suite. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gpgme1.0/+bug/2037595/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
