[issue29870] ssl socket leak

Changes by Yury Selivanov : -- pull_requests: +2111 ___ Python tracker ___ ___

[issue29870] ssl socket leak

Changes by Yury Selivanov : -- pull_requests: +2110 ___ Python tracker ___ ___

[issue29870] ssl socket leak

Changes by Yury Selivanov : -- pull_requests: +2109 ___ Python tracker ___ ___

[issue29870] ssl socket leak

Mariatta Wijaya added the comment: I believe this was fixed in https://bugs.python.org/issue29738, so I'm closing this. Thanks :) -- nosy: +Mariatta resolution: -> fixed stage: -> resolved status: open -> closed ___ Python tracker

[issue29870] ssl socket leak

Antoine Pitrou added the comment: > Antoine, you might find multissl.py helpful. Very nice, thank you! -- ___ Python tracker ___

[issue29870] ssl socket leak

Christian Heimes added the comment: Antoine, you might find multissl.py helpful. I wrote a script to automate testing with multiple versions of OpenSSL and libressl. The first time it takes about half an hour to download, compile and install all versions locally.

[issue29870] ssl socket leak

Changes by Christian Heimes : -- pull_requests: +1229 ___ Python tracker ___ ___

[issue29870] ssl socket leak

Christian Heimes added the comment: Yes, I'm currently testing the change with a bunch of OpenSSL and LibreSSL versions. By the way the memory issue can be reproduced with any certificate that contains a CRL distribution point. Letsencrypt certs don't have a CRL DP. I guess Alexander's test

[issue29870] ssl socket leak

Alexander Mohr added the comment: awesome! Thanks for finding a proposing fix pitrou! btw I found an example of freeing this structure here: http://www.zedwood.com/article/c-openssl-parse-x509-certificate-pem -- ___ Python tracker

[issue29870] ssl socket leak

Antoine Pitrou added the comment: So we should use it instead of sk_DIST_POINT_free()? I'd like to minimize potential breakage here. -- ___ Python tracker

[issue29870] ssl socket leak

Christian Heimes added the comment: CRL_DIST_POINTS_free() should be available in all supported OpenSSL versions. The function is defined by DECLARE_ASN1_FUNCTIONS(CRL_DIST_POINTS). -- ___ Python tracker

[issue29870] ssl socket leak

Antoine Pitrou added the comment: The following addition fixes the leak: diff --git a/Modules/_ssl.c b/Modules/_ssl.c index bb40051..8f5facd 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -1203,6 +1203,8 @@ _get_crl_dp(X509 *certificate) { Py_XDECREF(lst); #if OPENSSL_VERSION_NUMBER

[issue29870] ssl socket leak

Antoine Pitrou added the comment: Ok, thank you. I've tweaked the script to remove most threads and use maps.google.com (which is faster here), and I managed to bisect the leak to deduce that the offending changeset is 598894ff48e9c1171cb2ec1c798235826a75c7e0. --

[issue29870] ssl socket leak

Alexander Mohr added the comment: see graphs here: https://github.com/kennethreitz/requests/issues/3933, x-axis is number of requests not what it says (seconds). -- ___ Python tracker

[issue29870] ssl socket leak

Antoine Pitrou added the comment: Google is not very fast here (a couple of requests / sec at most). How many requests does it take to see a clear tendency? -- ___ Python tracker

[issue29870] ssl socket leak

Alexander Mohr added the comment: ya, my sample script hits google.com , it's pretty fast. It just does a "HEAD". > On Apr 11, 2017, at 9:14 AM, Antoine Pitrou wrote: > > > Antoine Pitrou added the comment: > > Is there a fast enough remote

[issue29870] ssl socket leak

Antoine Pitrou added the comment: Is there a fast enough remote server that shows the leak? I've tested with my own remote server (https://pitrou.net/), but it doesn't leak. -- ___ Python tracker

[issue29870] ssl socket leak

Alexander Mohr added the comment: the interesting part is it doesn't leak with a local https server, it appears to need to be a remove server. -- ___ Python tracker

[issue29870] ssl socket leak

Antoine Pitrou added the comment: After adapting your test script to run against a local openssl server (`openssl s_server -www`), I can't see a single leak: the process is peaking at 20860 KB RSS. This is with Python 3.5 tip. Does it need a specific server to test against to showcase the

[issue29870] ssl socket leak

Alexander Mohr added the comment: yes, in the gist I created you can switch between the various clients, by default right now it uses raw sockets. -- ___ Python tracker

[issue29870] ssl socket leak

Antoine Pitrou added the comment: I see. This may mean the leak is in memory that's not managed directly by Python (e.g. some OpenSSL structure). Is there a way to reproduce without third-party libraries such as requests? -- ___ Python tracker

[issue29870] ssl socket leak

Alexander Mohr added the comment: @pitrou: sys.getallocatedblocks does not seem to increase -- ___ Python tracker ___

[issue29870] ssl socket leak

Antoine Pitrou added the comment: Can you record sys.getallocatedblocks() to see whether it grows continuously? -- nosy: +pitrou ___ Python tracker ___

[issue29870] ssl socket leak

Changes by Michael Sghaier : -- pull_requests: +1218 ___ Python tracker ___ ___

[issue29870] ssl socket leak

Changes by Cory Benfield : -- nosy: +Lukasa ___ Python tracker ___ ___ Python-bugs-list

[issue29870] ssl socket leak

Alexander Mohr added the comment: interestingly the valgrind run doesn't show a leak in the profile -- ___ Python tracker ___

[issue29870] ssl socket leak

Alexander Mohr added the comment: adding valgrind log of 3.5.3 on debian: jessie -- Added file: http://bugs.python.org/file46750/valgrind.log.gz ___ Python tracker

[issue29870] ssl socket leak

Alexander Mohr added the comment: validated 3.6 in fedora is affected as well, see github bug for charts. So it seems all 3.5.3+ versions are affected. I'm guessing it was introduced in one of the SSL changes in 3.5.3: https://docs.python.org/3.5/whatsnew/changelog.html#python-3-5-3

[issue29870] ssl socket leak

New submission from Alexander Mohr: When upgrading to 3.5.3 we noticed that the requests module was leaking memory rather quickly. This led to me logging the issue: https://github.com/kennethreitz/requests/issues/3933. After more investigation I've found that the leak is caused by the raw