Context: Mostly by following the conversation/links in
https://github.com/psf/requests/pull/6667 import time regressions https://github.com/psf/requests/issues/6790 https://github.com/psf/requests/pull/6791 sslcontext in pool manager regression https://github.com/psf/requests/issues/6715 https://github.com/psf/requests/pull/6716 (dup) https://github.com/psf/requests/issues/6717 concurrency regression https://github.com/psf/requests/pull/6667#issuecomment-2176042565 https://github.com/psf/requests/issues/6726 revert planned https://github.com/psf/requests/pull/6767 https://github.com/psf/requests/pull/6781 Requests maintainer (nateprewitt): "[...] #6767 [Revert caching a default SSLContext] is likely the path forward at this point. We've found a number of sharp edges with the default caching behavior to a point it probably doesn't belong in Requests. ** Bug watch added: github.com/psf/requests/issues #6790 https://github.com/psf/requests/issues/6790 ** Bug watch added: github.com/psf/requests/issues #6715 https://github.com/psf/requests/issues/6715 ** Bug watch added: github.com/psf/requests/issues #6717 https://github.com/psf/requests/issues/6717 ** Bug watch added: github.com/psf/requests/issues #6726 https://github.com/psf/requests/issues/6726 ** Description changed: - WIP. + [Impact] + + * The Python Requests library has a performance degradation + with OpenSSL 3 in certificate verification when compared + to OpenSSL 1.1 in Focal, affecting HTTPS connections. + + * This more significantly impacts performance of concurrent + HTTPS requests, due to lock contention when loading the + certificate store from the system/root CA certificates. + + * This impact is due to the expensive/slow call to SSLContext's + `load_verify_locations()`, which is impacted by locking. + + * In the Python Requests 2.32 series, there is a patch to + reuse a global SSLContext across concurrent connections, + when possible, so that function is called _only once_. + + * This results in a significant performance improvement + for concurrent requests to HTTPS addresses. + + [Test Plan] + + * Test case in comment 1. Results in comments 2 and 3. + + [Other Info] + + * Upstream commit: + https://github.com/psf/requests/commit/9a40d1277807f0a4f26c9a37eea8ec90faa8aadc + + * Unfortunately, Python Requests upstream received reports + of regressions in other areas (not performance) due to + this change, as seen in the mentions to the PR#6667: + https://github.com/psf/requests/pull/6667 + + * Fortunately, most regressions have received fixes, and + should work correctly, however, there's an unaddressed + issue with concurrent changes to the SSLContext, if the + workload/python source code does that (not all code does). + + * As a result, upstream plans to revert that change, and + only look at that again with more comprehensive tests, + although that has not been merged for months (PR#6767): + https://github.com/psf/requests/pull/6767 + + * Ubuntu Oracular ships the performance improvent patch, + so theoretically this code is being exposed to users + already. + + * Therefore, this bug/patches plan to ship the improvement + in "opt-in" mode, so that it's available but *not enabled* + by default -- which allows for easy testing and disabling. + + * This would allow users whose python code/workloads are NOT + impacted by the concurrency regressions to benefit from a + significant performance improvement. + (Summary in comment 4, some context in comment 5.) ** Changed in: python-urllib3 (Ubuntu Jammy) Assignee: Mauricio Faria de Oliveira (mfo) => Ioanna Alifieraki (joalif) ** Changed in: requests (Ubuntu Jammy) Assignee: Mauricio Faria de Oliveira (mfo) => Ioanna Alifieraki (joalif) ** Changed in: requests (Ubuntu Noble) Assignee: Mauricio Faria de Oliveira (mfo) => Ioanna Alifieraki (joalif) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2091733 Title: Performance improvement for concurrent SSL connections To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/python-urllib3/+bug/2091733/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
