Re: [Async-sig] Cancelling SSL connection
Thanks Dima & Nathaniel. I opened an asyncio bug. ( http://bugs.python.org/issue30740) Cheers, Mark On Wed, Jun 21, 2017 at 6:47 PM, Nathaniel Smith wrote: > SSLObject.unwrap has the contract that if it finishes successfully, then > the SSL connection has been cleanly shut down and both sides remain in > sync, and can continue to use the socket in unencrypted mode. When asyncio > calls unwrap before the handshake has completed, then this contract is > impossible to fulfill, and raising an error is the right thing to do. So > imo the ssl module is correct here, and this is a (minor) bug in asyncio. > > On Jun 21, 2017 12:49 PM, "Dima Tisnek" wrote: > >> Looks like a bug in the `ssl` module, not `asyncio`. >> >> Refer to https://github.com/openssl/openssl/issues/710 >> IMO `ssl` module should be prepared for this. >> >> I'd say post a bug to cpython and see what core devs have to say about it >> :) >> Please note exact versions of python and openssl ofc. >> >> my 2c: openssl has been a moving target every so often, it's quite >> possible that this change in the API escaped the devs. >> >> On 21 June 2017 at 19:50, Mark E. Haase wrote: >> > (I'm not sure if this is a newbie question or a bug report or something >> in >> > between. I apologize in advance if its off-topic. Let me know if I >> should >> > post this somewhere else.) >> > >> > If a task is cancelled while SSL is being negotiated, then an SSLError >> is >> > raised, but there's no way (as far as I can tell) for the caller to >> catch >> > it. (The example below is pretty contrived, but in an application I'm >> > working on, the user can cancel downloads at any time.) Here's an >> example: >> > >> > import asyncio, random, ssl >> > >> > async def download(host): >> > ssl_context = ssl.create_default_context() >> > reader, writer = await asyncio.open_connection(host, 443, >> > ssl=ssl_context) >> > request = f'HEAD / HTTP/1.1\r\nHost: {host}\r\n\r\n' >> > writer.write(request.encode('ascii')) >> > lines = list() >> > while True: >> > newdata = await reader.readline() >> > if newdata == b'\r\n': >> > break >> > else: >> > lines.append(newdata.decode('utf8').rstrip('\r\n')) >> > return lines[0] >> > >> > async def main(): >> > while True: >> > task = asyncio.Task(download('www.python.org')) >> > await asyncio.sleep(random.uniform(0.0, 0.5)) >> > task.cancel() >> > try: >> > response = await task >> > print(response) >> > except asyncio.CancelledError: >> > print('request cancelled!') >> > except ssl.SSLError: >> > print('caught SSL error') >> > await asyncio.sleep(1) >> > >> > loop = asyncio.get_event_loop() >> > loop.run_until_complete(main()) >> > loop.close() >> > >> > Running this script yields the following output: >> > >> > HTTP/1.1 200 OK >> > request cancelled! >> > HTTP/1.1 200 OK >> > HTTP/1.1 200 OK >> > : SSL >> handshake >> > failed >> > Traceback (most recent call last): >> > File "/usr/lib/python3.6/asyncio/base_events.py", line 803, in >> > _create_connection_transport >> > yield from waiter >> > File "/usr/lib/python3.6/asyncio/tasks.py", line 304, in _wakeup >> > future.result() >> > concurrent.futures._base.CancelledError >> > >> > During handling of the above exception, another exception occurred: >> > >> > Traceback (most recent call last): >> > File "/usr/lib/python3.6/asyncio/sslproto.py", line 577, in >> > _on_handshake_complete >> > raise handshake_exc >> > File "/usr/lib/python3.6/asyncio/sslproto.py", line 638, in >> > _process_write_backlog >> > ssldata = self._sslpipe.shutdown(self._finalize) >> > File "/usr/lib/python3.6/asyncio/sslproto.py", line 155, in >> shutdown >> > ssldata, appdata = self.feed_ssldata(b'') >> > File "/usr/lib/python3.6/asyncio/sslproto.py", line 219, in >> > feed_ssldata >> > self._sslobj.unwrap() >> > File "/usr/lib/python3.6/ssl.py", line 692, in unwrap >> > return self._sslobj.shutdown() >> > ssl.SSLError: [SSL] shutdown while in init (_ssl.c:2299) >> > >> > Is this a bug that I should file, or is there some reason that it's >> intended >> > to work this way? I can work around it with asyncio.shield(), but I >> think I >> > would prefer for the asyncio/sslproto.py to catch the SSLError and >> ignore >> > it. Maybe I'm being short sighted. >> > >> > Thanks, >> > Mark >> > >> > ___ >> > Async-sig mailing list >> > Async-sig@python.org >> > https://mail.python.org/mailman/listinfo/async-sig >> > Code of Conduct: https://www.python.org/psf/codeofconduct/ >> > >> ___ >> Async
Re: [Async-sig] Cancelling SSL connection
SSLObject.unwrap has the contract that if it finishes successfully, then the SSL connection has been cleanly shut down and both sides remain in sync, and can continue to use the socket in unencrypted mode. When asyncio calls unwrap before the handshake has completed, then this contract is impossible to fulfill, and raising an error is the right thing to do. So imo the ssl module is correct here, and this is a (minor) bug in asyncio. On Jun 21, 2017 12:49 PM, "Dima Tisnek" wrote: > Looks like a bug in the `ssl` module, not `asyncio`. > > Refer to https://github.com/openssl/openssl/issues/710 > IMO `ssl` module should be prepared for this. > > I'd say post a bug to cpython and see what core devs have to say about it > :) > Please note exact versions of python and openssl ofc. > > my 2c: openssl has been a moving target every so often, it's quite > possible that this change in the API escaped the devs. > > On 21 June 2017 at 19:50, Mark E. Haase wrote: > > (I'm not sure if this is a newbie question or a bug report or something > in > > between. I apologize in advance if its off-topic. Let me know if I should > > post this somewhere else.) > > > > If a task is cancelled while SSL is being negotiated, then an SSLError is > > raised, but there's no way (as far as I can tell) for the caller to catch > > it. (The example below is pretty contrived, but in an application I'm > > working on, the user can cancel downloads at any time.) Here's an > example: > > > > import asyncio, random, ssl > > > > async def download(host): > > ssl_context = ssl.create_default_context() > > reader, writer = await asyncio.open_connection(host, 443, > > ssl=ssl_context) > > request = f'HEAD / HTTP/1.1\r\nHost: {host}\r\n\r\n' > > writer.write(request.encode('ascii')) > > lines = list() > > while True: > > newdata = await reader.readline() > > if newdata == b'\r\n': > > break > > else: > > lines.append(newdata.decode('utf8').rstrip('\r\n')) > > return lines[0] > > > > async def main(): > > while True: > > task = asyncio.Task(download('www.python.org')) > > await asyncio.sleep(random.uniform(0.0, 0.5)) > > task.cancel() > > try: > > response = await task > > print(response) > > except asyncio.CancelledError: > > print('request cancelled!') > > except ssl.SSLError: > > print('caught SSL error') > > await asyncio.sleep(1) > > > > loop = asyncio.get_event_loop() > > loop.run_until_complete(main()) > > loop.close() > > > > Running this script yields the following output: > > > > HTTP/1.1 200 OK > > request cancelled! > > HTTP/1.1 200 OK > > HTTP/1.1 200 OK > > : SSL > handshake > > failed > > Traceback (most recent call last): > > File "/usr/lib/python3.6/asyncio/base_events.py", line 803, in > > _create_connection_transport > > yield from waiter > > File "/usr/lib/python3.6/asyncio/tasks.py", line 304, in _wakeup > > future.result() > > concurrent.futures._base.CancelledError > > > > During handling of the above exception, another exception occurred: > > > > Traceback (most recent call last): > > File "/usr/lib/python3.6/asyncio/sslproto.py", line 577, in > > _on_handshake_complete > > raise handshake_exc > > File "/usr/lib/python3.6/asyncio/sslproto.py", line 638, in > > _process_write_backlog > > ssldata = self._sslpipe.shutdown(self._finalize) > > File "/usr/lib/python3.6/asyncio/sslproto.py", line 155, in > shutdown > > ssldata, appdata = self.feed_ssldata(b'') > > File "/usr/lib/python3.6/asyncio/sslproto.py", line 219, in > > feed_ssldata > > self._sslobj.unwrap() > > File "/usr/lib/python3.6/ssl.py", line 692, in unwrap > > return self._sslobj.shutdown() > > ssl.SSLError: [SSL] shutdown while in init (_ssl.c:2299) > > > > Is this a bug that I should file, or is there some reason that it's > intended > > to work this way? I can work around it with asyncio.shield(), but I > think I > > would prefer for the asyncio/sslproto.py to catch the SSLError and ignore > > it. Maybe I'm being short sighted. > > > > Thanks, > > Mark > > > > ___ > > Async-sig mailing list > > Async-sig@python.org > > https://mail.python.org/mailman/listinfo/async-sig > > Code of Conduct: https://www.python.org/psf/codeofconduct/ > > > ___ > Async-sig mailing list > Async-sig@python.org > https://mail.python.org/mailman/listinfo/async-sig > Code of Conduct: https://www.python.org/psf/codeofconduct/ > ___ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig C
Re: [Async-sig] Cancelling SSL connection
Looks like a bug in the `ssl` module, not `asyncio`. Refer to https://github.com/openssl/openssl/issues/710 IMO `ssl` module should be prepared for this. I'd say post a bug to cpython and see what core devs have to say about it :) Please note exact versions of python and openssl ofc. my 2c: openssl has been a moving target every so often, it's quite possible that this change in the API escaped the devs. On 21 June 2017 at 19:50, Mark E. Haase wrote: > (I'm not sure if this is a newbie question or a bug report or something in > between. I apologize in advance if its off-topic. Let me know if I should > post this somewhere else.) > > If a task is cancelled while SSL is being negotiated, then an SSLError is > raised, but there's no way (as far as I can tell) for the caller to catch > it. (The example below is pretty contrived, but in an application I'm > working on, the user can cancel downloads at any time.) Here's an example: > > import asyncio, random, ssl > > async def download(host): > ssl_context = ssl.create_default_context() > reader, writer = await asyncio.open_connection(host, 443, > ssl=ssl_context) > request = f'HEAD / HTTP/1.1\r\nHost: {host}\r\n\r\n' > writer.write(request.encode('ascii')) > lines = list() > while True: > newdata = await reader.readline() > if newdata == b'\r\n': > break > else: > lines.append(newdata.decode('utf8').rstrip('\r\n')) > return lines[0] > > async def main(): > while True: > task = asyncio.Task(download('www.python.org')) > await asyncio.sleep(random.uniform(0.0, 0.5)) > task.cancel() > try: > response = await task > print(response) > except asyncio.CancelledError: > print('request cancelled!') > except ssl.SSLError: > print('caught SSL error') > await asyncio.sleep(1) > > loop = asyncio.get_event_loop() > loop.run_until_complete(main()) > loop.close() > > Running this script yields the following output: > > HTTP/1.1 200 OK > request cancelled! > HTTP/1.1 200 OK > HTTP/1.1 200 OK > : SSL handshake > failed > Traceback (most recent call last): > File "/usr/lib/python3.6/asyncio/base_events.py", line 803, in > _create_connection_transport > yield from waiter > File "/usr/lib/python3.6/asyncio/tasks.py", line 304, in _wakeup > future.result() > concurrent.futures._base.CancelledError > > During handling of the above exception, another exception occurred: > > Traceback (most recent call last): > File "/usr/lib/python3.6/asyncio/sslproto.py", line 577, in > _on_handshake_complete > raise handshake_exc > File "/usr/lib/python3.6/asyncio/sslproto.py", line 638, in > _process_write_backlog > ssldata = self._sslpipe.shutdown(self._finalize) > File "/usr/lib/python3.6/asyncio/sslproto.py", line 155, in shutdown > ssldata, appdata = self.feed_ssldata(b'') > File "/usr/lib/python3.6/asyncio/sslproto.py", line 219, in > feed_ssldata > self._sslobj.unwrap() > File "/usr/lib/python3.6/ssl.py", line 692, in unwrap > return self._sslobj.shutdown() > ssl.SSLError: [SSL] shutdown while in init (_ssl.c:2299) > > Is this a bug that I should file, or is there some reason that it's intended > to work this way? I can work around it with asyncio.shield(), but I think I > would prefer for the asyncio/sslproto.py to catch the SSLError and ignore > it. Maybe I'm being short sighted. > > Thanks, > Mark > > ___ > Async-sig mailing list > Async-sig@python.org > https://mail.python.org/mailman/listinfo/async-sig > Code of Conduct: https://www.python.org/psf/codeofconduct/ > ___ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
[Async-sig] Cancelling SSL connection
(I'm not sure if this is a newbie question or a bug report or something in between. I apologize in advance if its off-topic. Let me know if I should post this somewhere else.) If a task is cancelled while SSL is being negotiated, then an SSLError is raised, but there's no way (as far as I can tell) for the caller to catch it. (The example below is pretty contrived, but in an application I'm working on, the user can cancel downloads at any time.) Here's an example: import asyncio, random, ssl async def download(host): ssl_context = ssl.create_default_context() reader, writer = await asyncio.open_connection(host, 443, ssl=ssl_context) request = f'HEAD / HTTP/1.1\r\nHost: {host}\r\n\r\n' writer.write(request.encode('ascii')) lines = list() while True: newdata = await reader.readline() if newdata == b'\r\n': break else: lines.append(newdata.decode('utf8').rstrip('\r\n')) return lines[0] async def main(): while True: task = asyncio.Task(download('www.python.org')) await asyncio.sleep(random.uniform(0.0, 0.5)) task.cancel() try: response = await task print(response) except asyncio.CancelledError: print('request cancelled!') except ssl.SSLError: print('caught SSL error') await asyncio.sleep(1) loop = asyncio.get_event_loop() loop.run_until_complete(main()) loop.close() Running this script yields the following output: HTTP/1.1 200 OK request cancelled! HTTP/1.1 200 OK HTTP/1.1 200 OK : SSL handshake failed Traceback (most recent call last): File "/usr/lib/python3.6/asyncio/base_events.py", line 803, in _create_connection_transport yield from waiter File "/usr/lib/python3.6/asyncio/tasks.py", line 304, in _wakeup future.result() concurrent.futures._base.CancelledError During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/asyncio/sslproto.py", line 577, in _on_handshake_complete raise handshake_exc File "/usr/lib/python3.6/asyncio/sslproto.py", line 638, in _process_write_backlog ssldata = self._sslpipe.shutdown(self._finalize) File "/usr/lib/python3.6/asyncio/sslproto.py", line 155, in shutdown ssldata, appdata = self.feed_ssldata(b'') File "/usr/lib/python3.6/asyncio/sslproto.py", line 219, in feed_ssldata self._sslobj.unwrap() File "/usr/lib/python3.6/ssl.py", line 692, in unwrap return self._sslobj.shutdown() ssl.SSLError: [SSL] shutdown while in init (_ssl.c:2299) Is this a bug that I should file, or is there some reason that it's intended to work this way? I can work around it with asyncio.shield(), but I think I would prefer for the asyncio/sslproto.py to catch the SSLError and ignore it. Maybe I'm being short sighted. Thanks, Mark ___ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/