Maxim Yegorushkin maxim.yegorush...@gmail.com added the comment:
I am still having this issue with Python-2.7.2 (which includes the patch) and
openssl-devel-1.0.0e-1.fc14.x86_64:
File ./download.py, line 195, in download_file
ftp.retrbinary(retr + filename, lambda data:
Antoine Pitrou pit...@free.fr added the comment:
Ok, I've backported the shutdown fixes to 2.6 (r80419) and 3.1 (r80420). Thanks
again everyone.
--
status: pending - closed
versions: +Python 2.6, Python 3.1
___
Python tracker rep...@bugs.python.org
Antoine Pitrou pit...@free.fr added the comment:
Patch committed to r80392 (trunk) and r80394 (py3k). Am pondering a partial
backport of the SSL shutdown fixes to 2.6 and 3.1 (test_ftplib doesn't need any
backport since it doesn't support TLS in these branches).
--
assignee: - pitrou
Antoine Pitrou pit...@free.fr added the comment:
With regards to SSL_peek() blocking, you'd need to explain yourself
better on that one. The patch has been tested with the test cases
from Python SVN enough to be happy they run ok.
IIRC it worked with OpenSSL 1.0.0 but failed with 0.9.8k.
I
Giampaolo Rodola' g.rod...@gmail.com added the comment:
If everyone agrees on error: [Errno 0] Error being a legitimate alias for a
connection closed event condition then I'd say the test server looks good,
even if I think that expecting a ssl.SSLError derived exception would have made
more
Antoine Pitrou pit...@free.fr added the comment:
If everyone agrees on error: [Errno 0] Error being a legitimate alias
for a connection closed event condition then I'd say the test server
looks good, even if I think that expecting a ssl.SSLError derived
exception would have made more
Darryl Miles darryl.mi...@darrylmiles.org added the comment:
With regards to create test cases for certain situations, sure this would be
possible but not with pure python since your APIs deny/inhibit the particular
things required to force a situation for a test case.
With regards to
Antoine Pitrou pit...@free.fr added the comment:
After thinking a bit about it, we're in quasi-feature freeze now for 2.x and it
doesn't sound reasonable to add options to unwrap(); we would need to write
tests to exercise these options and their behaviour. Also, they aren't strictly
needed
Darryl Miles darryl.mi...@darrylmiles.org added the comment:
I am unable to get make test to run from an unpatched version in SVN (details
below of make output).
Please find attached an updated patch for your consideration (and testing, as I
can't test it due to 'make test' SIGSEGV on CentOS
Antoine Pitrou pit...@free.fr added the comment:
To just run the test_ftplib tests, you can use:
$ ./python -m test.regrtest -v -uall test_ftplib
(Other tests involving SSL sockets are test_ssl, test_smtpnet,
test_imaplib and test_poplib)
1) Some thing that concern me, the unwrap()
Darryl Miles darryl.mi...@darrylmiles.org added the comment:
To explain why you need 2 modes, a client/server would expect to do the
following pseudo actions for maximum efficiency:
set_socket_timeout(600_SECONDS) # or useful default
send_data_over_ssl(QUIT\r\n)
Darryl Miles darryl.mi...@darrylmiles.org added the comment:
In order to build Python with a specific version of OpenSSL followed the CYGWIN
instructions and edited Modules/Setup to make it read (note - I added
-L$(SSL) into the linker options too, since by default on CentOS 5.4 i386
OpenSSL
Darryl Miles darryl.mi...@darrylmiles.org added the comment:
I've updated my attachment to the bug, if you read the old one please re-read
the updated version (since some points in there were not accurate).
With regards to the OpenSSL error return -1/ERROR_SYSCALL with errno==0 being
Changes by Darryl Miles darryl.mi...@darrylmiles.org:
Removed file: http://bugs.python.org/file16838/python_ssl.c.txt
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8108
___
Antoine Pitrou pit...@free.fr added the comment:
Ok, I raised the errno 0 issue on the openssl mailing-list, and I had a nice
response by Darryl Miles here:
http://www.mail-archive.com/openssl-us...@openssl.org/msg60710.html
The bottom line is that any socket error return from unwrap(), that
Antoine Pitrou pit...@free.fr added the comment:
My analysis of SSL_shutdown() was missing something: timeout sockets are really
non-blocking sockets in disguise. I guess it's never too late to notice.
OpenSSL doesn't know about the timeout, so we have to do it ourselves.
Ironically, the
Antoine Pitrou pit...@free.fr added the comment:
Ok, here is a patch fixing all test_ftplib failures (tested with openssl 0.9.8k
and 1.0.0).
There are a couple of fixes here:
- a workaround for the strange SSL_shutdown() error return, also witnessed by
LightHTTPd users (see URL in comments)
-
Changes by Antoine Pitrou pit...@free.fr:
Added file: http://bugs.python.org/file16778/newssl.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8108
___
Giampaolo Rodola' billiej...@users.sourceforge.net added the comment:
def handle_error(self):
-raise
+# Ignore errors while closing, because the remote end could have
+# abruptly shut down the TCP connection while we are still
+# waiting for SSL
Antoine Pitrou pit...@free.fr added the comment:
What happens if you move the SSL shutdown() call in handle_close()
method and leave handle_error() unchanged?
I get occasional crashes such as the following:
Exception in thread Thread-60:
Traceback (most recent call last):
File
Antoine Pitrou pit...@free.fr added the comment:
By the way, this broken pipe error could be due to the fact that the
FTP_TLS client never tries to call unwrap() on its SSL socket. Perhaps
the close() method should be overriden?
--
___
Python tracker
Antoine Pitrou pit...@free.fr added the comment:
Here is a new patch. It doesn't change handle_error() but instead ignores
abruptly closed sockets when doing the SSL shutdown.
The underlying problem seems to be an asyncore limitation: close() can be
called after an error such as EPIPE or
Giampaolo Rodola' billiej...@users.sourceforge.net added the comment:
By the way, this broken pipe error could be due to the fact that the
FTP_TLS client never tries to call unwrap() on its SSL socket. Perhaps
the close() method should be overriden?
ftplib.FTP_TLS class already calls
Antoine Pitrou pit...@free.fr added the comment:
As for our test server we are currently shutting down the SSL layer for
both connections (control and data) so this might be the cause of EPIPE.
I don't understand your reasoning. These are separate connections, closing one
shouldn't affect
Giampaolo Rodola' billiej...@users.sourceforge.net added the comment:
I don't understand your reasoning. These are separate connections,
closing one shouldn't affect the other.
I meant another thing. I was talking about the fact that the test server
attempts to shutdown() both control and
Antoine Pitrou pit...@free.fr added the comment:
- ftplib.FTP_TLS.quit: I wouldn't call unwrap() here as RFC-4217 does
that only when dealing with CCC command which is not implemented by
ftplib.py.
Ok.
- I'd be for calling _do_ssl_shutdown() from handle_close() instead of
from close()
Antoine Pitrou pit...@free.fr added the comment:
New patch without the quit() method change.
--
Added file: http://bugs.python.org/file16781/newssl3.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8108
Changes by Jesús Cea Avión j...@jcea.es:
--
nosy: +jcea
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8108
___
___
Python-bugs-list mailing list
Giampaolo Rodola' billiej...@users.sourceforge.net added the comment:
Yes, it's the _sslobj.shutdow() call:
File test_ftplib.py, line 332, in handle_close
self.socket = self.socket.unwrap()
File /usr/local/lib/python2.7/ssl.py, line 258, in unwrap
s = self._sslobj.shutdown()
error:
Changes by Florent Xicluna florent.xicl...@gmail.com:
--
nosy: +jcea
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8108
___
___
Python-bugs-list
Antoine Pitrou pit...@free.fr added the comment:
Since it's not clear to me where exactly this comes from, whether it's
from the Python C binding or OpenSSL itself, I tried to put some
debugging printf() calls in Modules/_ssl.c, but it seems that after
installing OpenSSL 0.9.8m I'm no
Giampaolo Rodola' billiej...@users.sourceforge.net added the comment:
No I haven't, but I tried just now and I get the same error.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8108
___
Antoine Pitrou pit...@free.fr added the comment:
No I haven't, but I tried just now and I get the same error.
I think inflate is a function exported by the zlib. Perhaps you can
add -lz to the linking flags.
(Googling hints that OpenSSL can depend on the zlib if compression is
enabled)
Giampaolo Rodola' billiej...@users.sourceforge.net added the comment:
As suggested in this thread:
http://mirt.net/pipermail/stunnel-users/2005-July/000661.html
...I made the following change to the Makefile:
- LIBS= -lpthread -ldl -lutil
+ LIBS= -lpthread -ldl -lutil -lz
That
Antoine Pitrou pit...@free.fr added the comment:
I've tried to build against OpenSSL 1.0.0, and I get further failures in
test_ssl. Since I don't know whether they are related, I've created a separate
issue for them: issue8322
--
___
Python tracker
Changes by Benjamin Peterson benja...@python.org:
--
priority: high - release blocker
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8108
___
___
Antoine Pitrou pit...@free.fr added the comment:
I don't think this should be a release blocker for the beta, although I agree
it should be one for the final release (and RCs).
By the way, it doesn't affect maintenance branches.
--
versions: -Python 2.6, Python 3.1
Antoine Pitrou pit...@free.fr added the comment:
Giampaolo, which call exactly triggers the error: [Errno 0] Error?
It doesn't seem it can be _sslobj.shutdown itself.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8108
Changes by Benjamin Peterson benja...@python.org:
--
priority: release blocker - critical
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8108
___
Giampaolo Rodola' billiej...@users.sourceforge.net added the comment:
@Antoine Pitrou: ok, I think I'll have to be able to replicate the error in
order to try to fix this. I'm gonna try on FreeBSD and another Linux box later
today.
--
___
Python
Florent Xicluna florent.xicl...@gmail.com added the comment:
I don't know if it is better to fix it in the ftplib or the ssl module.
The previous patch fixes it at the SSL level (stdlib only).
This patch implements the fix at the ftplib level (stdlib + test).
--
Added file:
Antoine Pitrou pit...@free.fr added the comment:
Florent, you shouldn't need to silence socket.error at this level.
I think what needs to happen is that, when unwrap() fails in
SSLConnection.close(), handle_read() and handle_write() retry the unwrap()
call. That's what I gather from the
Florent Xicluna florent.xicl...@gmail.com added the comment:
You're right, there's no need to catch the socket error in ftplib module.
Here is the result of the debugging patch in the console:
$ ./python -m test.regrtest -uall test_ftplib
test_ftplib
success
success
success
success
success
Changes by Florent Xicluna florent.xicl...@gmail.com:
Removed file: http://bugs.python.org/file16641/issue8108_ftplib_ssl_098m.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8108
___
Giampaolo Rodola' billiej...@users.sourceforge.net added the comment:
Florent can you give me a clue on how to make python use OpenSSL 0.9.8k on
Ubuntu?
Despite I compiled and installed 0.9.8k from sources the system version keeps
being 0.9.8g and I guess that Python is using that one.
Do I
Florent Xicluna florent.xicl...@gmail.com added the comment:
Patch ftplib/test_ftplib updated.
It is as before (0.9.8k): SSL_ERROR_WANT_READ is ignored on connection unwrap().
--
Added file: http://bugs.python.org/file16648/issue8108_ftplib_ssl_098m_v2.diff
Florent Xicluna florent.xicl...@gmail.com added the comment:
Giampaolo, IMHO you need 0.9.8m to reproduce the issue.
I installed it from the Debian testing repo (I'm on debian).
~ $ apt-cache policy openssl
openssl:
Installed : 0.9.8m-2
Candidate : 0.9.8m-2
Table de version :
*** 0.9.8m-2
Giampaolo Rodola' billiej...@users.sourceforge.net added the comment:
Ok, I've finally been able to reproduce the issue by installing OpenSSL 0.9.8m
on my Ubuntu box.
By doing some debugging I'm starting to think that maybe there's something
wrong with the shutdown() method implementation
Antoine Pitrou pit...@free.fr added the comment:
Giampaolo, I tried committing your patch but the buildbot still fails, with the
following errors:
test_auth_issued_twice (test.test_ftplib.TestTLS_FTPClass) ... ok
test_auth_ssl (test.test_ftplib.TestTLS_FTPClass) ... ok
test_control_connection
Changes by Antoine Pitrou pit...@free.fr:
--
stage: test needed - needs patch
versions: +Python 2.6, Python 3.1
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8108
___
Florent Xicluna florent.xicl...@gmail.com added the comment:
This patch solves the problem with 0.9.8m on my Debian workstation.
Actually there's 4 kind of errors on shutdown():
- SSL_ERROR_WANT_READ
- SSL_ERROR_WANT_WRITE
- socket.error: errno.PIPE (32)
- socket.error: 0
Since the command
Antoine Pitrou pit...@free.fr added the comment:
Ok, I finally agree that test_ftplib should be patched.
The patch isn't exactly correct: it should ideally retry the unwrap() call
later, rather than simply ignore the error. But since it's just used for
testing, it looks sufficient.
(we
Giampaolo Rodola' billiej...@users.sourceforge.net added the comment:
The patch isn't exactly correct: it should ideally retry the unwrap()
call later, rather than simply ignore the error. But since it's just
used for testing, it looks sufficient.
Actually unwrap() should already be
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com:
--
nosy: +Arfrever
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8108
___
Bill Janssen bill.jans...@gmail.com added the comment:
AFAICT, his proposed quick fix snippet should be good enough for us.
Depends on what we want. It just suppresses information that's now available.
What we'd really like is for the caller to recognize that close() can fail, and
should
STINNER Victor victor.stin...@haypocalc.com added the comment:
close() can fail
Not close, unwrap() (shutdown()) ;-)
Again, why does this failure only show up in the FTP test?
Maybe because only the FTP test uses an SSL socket in non blocking mode.
--
Antoine Pitrou pit...@free.fr added the comment:
Depends on what we want. It just suppresses information that's now
available. What we'd really like is for the caller to recognize that
close() can fail, and should be re-tried if it does. That requires
that we signal the error back up and
Bill Janssen bill.jans...@gmail.com added the comment:
Maybe because only the FTP test uses an SSL socket in non blocking mode.
No, the SSL unit tests also do this. I think Giampaolo is right, what we're
seeing is bad error handling in the FTP test code.
--
Bill Janssen bill.jans...@gmail.com added the comment:
Depends on what we want. It just suppresses information that's now
available. What we'd really like is for the caller to recognize that
close() can fail, and should be re-tried if it does. That requires
that we signal the error back
Bill Janssen bill.jans...@gmail.com added the comment:
Victor, I'll take a look.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8108
___
___
Bill Janssen bill.jans...@gmail.com added the comment:
What's happening is that the new state returns from SSL_shutdown() are saying,
the shutdown you asked for didn't happen this time, but call me again when you
get a chance. And here is a hint about why it didn't happen, so that if you
are
Giampaolo Rodola' billiej...@users.sourceforge.net added the comment:
I'm not sure this should be handled in ssl.py.
ssl.py correctly raises ERROR_WANT_READ/WRITE if the shutdown operation didn't
complete and that's ok.
It should be up to the upper application (in our case asyncore) to deal
Bill Janssen bill.jans...@gmail.com added the comment:
Looking at this further, what we're seeing is the trace of an ineffective
attempt to handle an exception presumably raised from the FTP code. Can we see
that exception? What's the actual state of the TCP connection at this point?
Bill Janssen bill.jans...@gmail.com added the comment:
I think Giampaolo is right about this not being an ssl.py issue. Could the
exception handling in ftp.py perhaps also be made more sophisticated? I'd
really love to understand what the state of the TCP connection is here. I'm
presuming
Bill Janssen bill.jans...@gmail.com added the comment:
And it would be interesting to know why all the SSL module tests don't fail in
the same way.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8108
Giampaolo Rodola' billiej...@users.sourceforge.net added the comment:
I'd really love to understand what the state of the TCP connection
is here. I'm presuming that it's still open, because otherwise you'd
get a different error from OpenSSL. So what's the error that's
triggering this
STINNER Victor victor.stin...@haypocalc.com added the comment:
I posted on openssl-users mailing list. The author of patch (change
SSL_shutdown() behaviour for non blocking socket) replied!
http://marc.info/?l=openssl-usersm=126838806919896w=2
--
Antoine Pitrou pit...@free.fr added the comment:
I posted on openssl-users mailing list. The author of patch (change
SSL_shutdown() behaviour for non blocking socket) replied!
http://marc.info/?l=openssl-usersm=126838806919896w=2
Interesting read, thank you.
AFAICT, his proposed quick fix
STINNER Victor victor.stin...@haypocalc.com added the comment:
The problem comes from SSL_Shutdown() on a non blocking connection. See:
http://www.mail-archive.com/openssl-...@openssl.org/msg24097.html
--
___
Python tracker rep...@bugs.python.org
STINNER Victor victor.stin...@haypocalc.com added the comment:
The patch changing SSL_Shutdown() behaviour on non blocking socket was written
in 2006 by Darryl Miles and it will commited into 0.9.8m. The original patch:
http://marc.info/?l=openssl-devm=115153998821797w=2
In our testcase,
STINNER Victor victor.stin...@haypocalc.com added the comment:
More informations:
http://www.openssl.org/docs/ssl/SSL_shutdown.html
http://rt.openssl.org/Ticket/Display.html?user=guestpass=guestid=1891
Extract of the manual page:
If the underlying BIO is non-blocking, SSL_shutdown() will
Changes by Antoine Pitrou pit...@free.fr:
--
nosy: +janssen
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8108
___
___
Python-bugs-list mailing
STINNER Victor victor.stin...@haypocalc.com added the comment:
There are different kind of bugs, but all of them are related to
SSL_Shutdown(). It depends on the socket timeout:
- blocking socket: nothing special (no change from previous OpenSSL version)
- timeout or non blocking socket: if
New submission from Florent Xicluna florent.xicl...@gmail.com:
There's such failure on trunk and py3k when openssl 0.9.8m is installed.
(Debian platform)
No failure with 0.9.8k.
test_ftplib
Exception in thread Thread-40:
Traceback (most recent call last):
File ./Lib/threading.py, line 530, in
Changes by Florent Xicluna florent.xicl...@gmail.com:
--
nosy: +giampaolo.rodola, haypo, pitrou
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8108
___
Florent Xicluna florent.xicl...@gmail.com added the comment:
It occurs only for trunk and 3.x, where the ftp over ssl is implemented. See
#2054.
And the Debian alpha buildbots have the same library 0.9.8m:
http://www.python.org/dev/buildbot/all/builders/alpha%20Debian%20trunk
Florent Xicluna florent.xicl...@gmail.com added the comment:
OpenSSL changelog (scroll to 0.9.8m): http://www.openssl.org/news/changelog.html
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8108
77 matches
Mail list logo
