[issue23749] asyncio missing wrap_socket (starttls)

[STINNER Victor]

STINNER Victor  added the comment:

I found a race condition in START TLS: bpo-33674. I'm fixing it (I'm just 
waiting to merge my PR which has already been approved).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Yury Selivanov]

Change by Yury Selivanov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Is the issue done?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Yury Selivanov]

Yury Selivanov  added the comment:

I have a feeling that using threads+IO+asyncio makes the test too unstable on 
some Windows buildbots.  I'll rewrite start-tls tests without using threads.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[STINNER Victor]

STINNER Victor  added the comment:

test_start_tls_server_1() still fails randomly. Example on AppVeyor on my PR 
5423:

https://ci.appveyor.com/project/python/cpython/build/3.7build11472

ERROR: test_start_tls_server_1 
(test.test_asyncio.test_sslproto.SelectorStartTLSTests)
--
Traceback (most recent call last):
  File "C:\projects\cpython\lib\test\test_asyncio\test_sslproto.py", line 293, 
in test_start_tls_server_1
asyncio.wait_for(main(), loop=self.loop, timeout=10))
  File "C:\projects\cpython\lib\asyncio\base_events.py", line 564, in 
run_until_complete
raise RuntimeError('Event loop stopped before Future completed.')
RuntimeError: Event loop stopped before Future completed.

I also had this failure on my Windows 10 VM when running "python -m test -R 3:3 
-v test_asyncio". I skipped the test to be able to debug bpo-32710.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Yury Selivanov]

Yury Selivanov  added the comment:


New changeset 1e5b25b8c0c45ccfd58da2cb82fdf231c6823fef by Yury Selivanov in 
branch 'master':
bpo-23749: Make start-tls tests more stable on win7 buildbot (GH-5409)
https://github.com/python/cpython/commit/1e5b25b8c0c45ccfd58da2cb82fdf231c6823fef


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Yury Selivanov]

Change by Yury Selivanov :


--
pull_requests: +5242
stage: resolved -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[STINNER Victor]

STINNER Victor  added the comment:

> Tests fail on x86 Windows7 3.x:

I created a more specific issue: bpo-32645, test_asyncio: TLS tests fail on 
"x86 Windows7" buildbot.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[STINNER Victor]

STINNER Victor  added the comment:

SelectorStartTLSTests failed once on x86 Tiger 3.x in build 453, but then 
passed, no idea why.

http://buildbot.python.org/all/#/builders/30/builds/453

==
ERROR: test_start_tls_client_1 
(test.test_asyncio.test_sslproto.SelectorStartTLSTests)
--
Traceback (most recent call last):
  File 
"/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/test/test_asyncio/test_sslproto.py",
 line 225, in test_start_tls_client_1
asyncio.wait_for(client(srv.addr), loop=self.loop, timeout=10))
  File 
"/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/asyncio/base_events.py", line 
440, in run_until_complete
return future.result()
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/asyncio/tasks.py", line 
398, in wait_for
raise futures.TimeoutError()
concurrent.futures._base.TimeoutError
==
ERROR: test_start_tls_server_1 
(test.test_asyncio.test_sslproto.SelectorStartTLSTests)
--
Traceback (most recent call last):
  File 
"/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/test/test_asyncio/test_sslproto.py",
 line 285, in test_start_tls_server_1
asyncio.wait_for(main(), loop=self.loop, timeout=10))
  File 
"/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/asyncio/base_events.py", line 
440, in run_until_complete
return future.result()
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/asyncio/tasks.py", line 
398, in wait_for
raise futures.TimeoutError()
concurrent.futures._base.TimeoutError

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[STINNER Victor]

STINNER Victor  added the comment:

http://buildbot.python.org/all/#/builders/58/builds/435

Tests fail on x86 Windows7 3.x:

==
ERROR: test_start_tls_client_1 
(test.test_asyncio.test_sslproto.ProactorStartTLSTests)
--
Traceback (most recent call last):
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\test_asyncio\test_sslproto.py",
 line 225, in test_start_tls_client_1
asyncio.wait_for(client(srv.addr), loop=self.loop, timeout=10))
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\asyncio\base_events.py",
 line 440, in run_until_complete
return future.result()
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\asyncio\tasks.py", 
line 398, in wait_for
raise futures.TimeoutError()
concurrent.futures._base.TimeoutError
==
ERROR: test_start_tls_client_1 
(test.test_asyncio.test_sslproto.SelectorStartTLSTests)
--
Traceback (most recent call last):
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\test_asyncio\test_sslproto.py",
 line 225, in test_start_tls_client_1
asyncio.wait_for(client(srv.addr), loop=self.loop, timeout=10))
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\asyncio\base_events.py",
 line 440, in run_until_complete
return future.result()
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\asyncio\tasks.py", 
line 398, in wait_for
raise futures.TimeoutError()
concurrent.futures._base.TimeoutError

--
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Yury Selivanov]

Yury Selivanov  added the comment:

> I think the feature is significant enough for a What's New entry.

Sure, Elvis and I will go through all NEWS items when it's time for what's new 
;)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

@yselivanov - thanks for adding this, it's a huge win.

I think the feature is significant enough for a What's New entry.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Yury Selivanov]

Change by Yury Selivanov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Yury Selivanov]

Yury Selivanov  added the comment:


New changeset f111b3dcb414093a4efb9d74b69925e535ddc470 by Yury Selivanov in 
branch 'master':
bpo-23749: Implement loop.start_tls() (#5039)
https://github.com/python/cpython/commit/f111b3dcb414093a4efb9d74b69925e535ddc470


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Yury Selivanov]

Change by Yury Selivanov :


--
pull_requests: +4923
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Christian Heimes]

Christian Heimes added the comment:

I'm removing myself and drop the SSL component. It's really a feature request 
for asyncio.

--
assignee: christian.heimes -> yselivanov
components:  -SSL

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

I'm very interested in this because, even though we do support STARTTLS in 
aiosmtpd, it's a hack using non-public symbols, and we have a hidden traceback! 
 (I.e. one that doesn't cause the test suite to fail, but only shows up when 
clients disconnect.)

Here's our STARTTLS implementation (at least as of this writing): 

https://github.com/aio-libs/aiosmtpd/blob/master/aiosmtpd/smtp.py#L361

And here's the bug description:

https://github.com/aio-libs/aiosmtpd/issues/83

We're getting eof_received() *after* connection_lost()!

And the "fix":

https://github.com/aio-libs/aiosmtpd/pull/101/files

Basically, once we flip the protocol to the SSLProtocol and then munge the 
transport, we have to keep the original transport around so that we can close 
that explicitly on connection_lost().

I don't really know whether this is 1) the right way to implement STARTTLS, and 
2) to handle the traceback fix given the APIs we have to work with today 
(Python 3.4-3.6).  But that's the problem right? :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Barry A. Warsaw]

Changes by Barry A. Warsaw :


--
nosy: +barry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Guido van Rossum]

Changes by Guido van Rossum :


--
nosy:  -gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[François]

Changes by François :


--
nosy: +Frzk

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Yury Selivanov]

Yury Selivanov added the comment:

With the latest change it's possible to implement starttls
as a separate package on PyPI, or even by copying/pasting a small
snipped of code in your project.

It's expected that we'll figure out the API design for starttls
during 3.6, so that we can add it in 3.7.

This issue should be kept open until we have a full public API
for starttls in asyncio.

--
versions:  -Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Roundup Robot]

Roundup Robot added the comment:

New changeset 3771a6326725 by Yury Selivanov in branch '3.5':
asyncio: Add "call_connection_made" arg to SSLProtocol.__init__
https://hg.python.org/cpython/rev/3771a6326725

New changeset 3e6739e5c2d0 by Yury Selivanov in branch '3.6':
Merge 3.5 (issue #23749)
https://hg.python.org/cpython/rev/3e6739e5c2d0

New changeset f2204eaba685 by Yury Selivanov in branch 'default':
Merge 3.6 (issue #23749)
https://hg.python.org/cpython/rev/f2204eaba685

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Alex Grönholm]

Alex Grönholm added the comment:

So is this going to make it into 3.6...?

--
nosy: +Alex Grönholm

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Christian Heimes]

Changes by Christian Heimes :


--
assignee:  -> christian.heimes
components: +SSL
nosy: +christian.heimes
versions: +Python 3.6, Python 3.7 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Yury Selivanov]

Yury Selivanov added the comment:

> yuri, did you saw guido added review on your patch?

Yes.  There are few more issues with the patch that I want to resolve before 
re-submitting it for another review.  Will do it soon.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[박세원]

박세원 added the comment:

https://bugs.python.org/review/23749/#msg1
yuri, did you saw guido added review on your patch?

--
nosy: +박세원

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Yury Selivanov]

Yury Selivanov added the comment:

I'll create a PR on the GitHub for this.  I like the proposed design, and I've 
implemented an SSL test micro-framework that we can use to test starttls in 
asyncio.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Robert Siemer]

Changes by Robert Siemer :


--
nosy: +siemer

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Yury Selivanov]

Yury Selivanov added the comment:

Guido, Victor, any thoughts about the (proto-)patch?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Yury Selivanov]

Yury Selivanov added the comment:

Guido, Victor,

Please find attached a first draft of the patch.  It's a very early attempt 
(i.e. I'm not including unit tests/docstrings), and its primary purpose is to 
gather initial feedback.

Some points:

1. As discussed earlier, the primary API point is new transports.TLSTransport 
class with a `start_tls(sslcontetx, *, server_side=False, 
server_hostname=None)` method.

2. While experimenting with the code and unit tests, I thought that it would be 
great if stream writers could do start_tls too, this patch has that too.  I 
like this new idea -- makes it so much simpler to write protocols.

--
keywords: +patch
Added file: http://bugs.python.org/file40866/tls1.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Yury Selivanov]

Yury Selivanov added the comment:

Here's an example client implementation with writer.start_tls() (taken from my 
debug code):

@asyncio.coroutine
def client(addr):
reader, writer = yield from asyncio.open_connection(
*addr, loop=loop)

print("CLIENT: ", (yield from reader.readexactly(4)))
writer.write(b'ehlo')
yield from writer.start_tls(sslctx)
# encrypted channel from this point
print("CLIENT: ", (yield from reader.readexactly(4)))

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Elizabeth Myers]

Elizabeth Myers added the comment:

> Therefore, the most convenient place to add the new API are *transports*.

I had an inkling this was the case, but I didn't know how to go about the 
creation of a new protocol and transport pair.

> I'm hesitant to add this API to Transport; it somehow feels wrong to put such 
> an implementation-specific thing there. E.g. I presume you can't do this for 
> an UDP transport.

DTLS (basically TLS over any datagram-oriented protocol, including UDP, SCTP, 
etc.) exists, so this makes sense, although I don't know if asyncio supports 
it, but the only major protocol I can think of that uses DTLS is WebRTC.

In any case, it could potentially make sense for other transport types, if not 
now, then in the future.

--
nosy: +Elizacat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Mathieu Sornay]

Changes by Mathieu Sornay :


--
nosy: +msornay

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Antoine Pitrou]

Changes by Antoine Pitrou pit...@free.fr:


--
nosy:  -pitrou

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23749
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Alex Grönholm]

Changes by Alex Grönholm alex.gronholm+pyt...@nextday.fi:


--
nosy: +alex.gronholm

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23749
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Yury Selivanov]

Yury Selivanov added the comment:

I'm working on porting pypostgresql (pure python postgresql driver) library to 
use asyncio as its underlying IO machinery.  And it appears that PQ3 protocol 
starts as clear text, and then upgrades to use TLS (if server or connection 
configured so).

I've been experimenting with various approaches to how we can design an API for 
this, and below are some of my thoughts:

1. The API cannot be implemented on event loop. Loops generally know nothing 
about the internal structure of transports, i.e. what loop or protocol the 
transport is attached to.

2. The API cannot be implemented on protocols. Protocols are decoupled from 
transports (they only receive a reference to the corresponding transport in 
their connection_made method). Access to the transport is requires to create an 
SSL proxy transport/protocol pair.

3. Therefore, the most convenient place to add the new API are *transports*. I 
propose to add a 'start_ssl' method to transports with the following signature:

def start_ssl(self, sslcontext=None,
  server_side=False, server_hostname=None) - Transport:

It will only be implemented on Python 3.5 (because of SSL MemoryBIO 
requirement).

Protocols can call 'start_ssl' any time after 'connection_made' is called. 
'start_ssl' returns a new Transport (ssl proxy) that has to be used from that 
moment on.  In case the SSL handshake fails, protocol's 'connection_lost' 
method will be called.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23749
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[Elizabeth Myers]

Changes by Elizabeth Myers elizab...@interlinked.me:


--
nosy:  -Elizacat

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23749
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket (starttls)

[STINNER Victor]

Changes by STINNER Victor victor.stin...@gmail.com:


--
title: asyncio missing wrap_socket - asyncio missing wrap_socket (starttls)

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23749
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com