[issue23749] asyncio missing wrap_socket (starttls)

STINNER Victor added the comment: I found a race condition in START TLS: bpo-33674. I'm fixing it (I'm just waiting to merge my PR which has already been approved). -- ___ Python tracker ___

[issue23749] asyncio missing wrap_socket (starttls)

Change by Yury Selivanov : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___ ___

[issue23749] asyncio missing wrap_socket (starttls)

Andrew Svetlov added the comment: Is the issue done? -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https

[issue23749] asyncio missing wrap_socket (starttls)

Yury Selivanov added the comment: I have a feeling that using threads+IO+asyncio makes the test too unstable on some Windows buildbots. I'll rewrite start-tls tests without using threads. -- ___ Python tracker

[issue23749] asyncio missing wrap_socket (starttls)

STINNER Victor added the comment: test_start_tls_server_1() still fails randomly. Example on AppVeyor on my PR 5423: https://ci.appveyor.com/project/python/cpython/build/3.7build11472 ERROR: test_start_tls_server_1 (test.test_asyncio.test_sslproto.SelectorStartTLSTests) -

[issue23749] asyncio missing wrap_socket (starttls)

Yury Selivanov added the comment: New changeset 1e5b25b8c0c45ccfd58da2cb82fdf231c6823fef by Yury Selivanov in branch 'master': bpo-23749: Make start-tls tests more stable on win7 buildbot (GH-5409) https://github.com/python/cpython/commit/1e5b25b8c0c45ccfd58da2cb82fdf231c6823fef --

[issue23749] asyncio missing wrap_socket (starttls)

Change by Yury Selivanov : -- pull_requests: +5242 stage: resolved -> patch review ___ Python tracker ___ ___ Python-bugs-list mailin

[issue23749] asyncio missing wrap_socket (starttls)

STINNER Victor added the comment: > Tests fail on x86 Windows7 3.x: I created a more specific issue: bpo-32645, test_asyncio: TLS tests fail on "x86 Windows7" buildbot. -- ___ Python tracker ___

[issue23749] asyncio missing wrap_socket (starttls)

STINNER Victor added the comment: SelectorStartTLSTests failed once on x86 Tiger 3.x in build 453, but then passed, no idea why. http://buildbot.python.org/all/#/builders/30/builds/453 == ERROR: test_start_tls_client_1 (test

[issue23749] asyncio missing wrap_socket (starttls)

STINNER Victor added the comment: http://buildbot.python.org/all/#/builders/58/builds/435 Tests fail on x86 Windows7 3.x: == ERROR: test_start_tls_client_1 (test.test_asyncio.test_sslproto.ProactorStartTLSTests) -

[issue23749] asyncio missing wrap_socket (starttls)

Yury Selivanov added the comment: > I think the feature is significant enough for a What's New entry. Sure, Elvis and I will go through all NEWS items when it's time for what's new ;) -- ___ Python tracker

[issue23749] asyncio missing wrap_socket (starttls)

Barry A. Warsaw added the comment: @yselivanov - thanks for adding this, it's a huge win. I think the feature is significant enough for a What's New entry. -- ___ Python tracker _

[issue23749] asyncio missing wrap_socket (starttls)

Change by Yury Selivanov : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___ ___

[issue23749] asyncio missing wrap_socket (starttls)

Yury Selivanov added the comment: New changeset f111b3dcb414093a4efb9d74b69925e535ddc470 by Yury Selivanov in branch 'master': bpo-23749: Implement loop.start_tls() (#5039) https://github.com/python/cpython/commit/f111b3dcb414093a4efb9d74b69925e535ddc470 -- _

[issue23749] asyncio missing wrap_socket (starttls)

Change by Yury Selivanov : -- pull_requests: +4923 stage: needs patch -> patch review ___ Python tracker ___ ___ Python-bugs-list mai

[issue23749] asyncio missing wrap_socket (starttls)

Christian Heimes added the comment: I'm removing myself and drop the SSL component. It's really a feature request for asyncio. -- assignee: christian.heimes -> yselivanov components: -SSL ___ Python tracker

[issue23749] asyncio missing wrap_socket (starttls)

Barry A. Warsaw added the comment: I'm very interested in this because, even though we do support STARTTLS in aiosmtpd, it's a hack using non-public symbols, and we have a hidden traceback! (I.e. one that doesn't cause the test suite to fail, but only shows up when clients disconnect.) Here'

[issue23749] asyncio missing wrap_socket (starttls)

Changes by Barry A. Warsaw : -- nosy: +barry ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.pytho

[issue23749] asyncio missing wrap_socket (starttls)

Changes by Guido van Rossum : -- nosy: -gvanrossum ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mai

[issue23749] asyncio missing wrap_socket (starttls)

Changes by François : -- nosy: +Frzk ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/ma

[issue23749] asyncio missing wrap_socket (starttls)

Yury Selivanov added the comment: With the latest change it's possible to implement starttls as a separate package on PyPI, or even by copying/pasting a small snipped of code in your project. It's expected that we'll figure out the API design for starttls during 3.6, so that we can add it in 3.7

[issue23749] asyncio missing wrap_socket (starttls)

Roundup Robot added the comment: New changeset 3771a6326725 by Yury Selivanov in branch '3.5': asyncio: Add "call_connection_made" arg to SSLProtocol.__init__ https://hg.python.org/cpython/rev/3771a6326725 New changeset 3e6739e5c2d0 by Yury Selivanov in branch '3.6': Merge 3.5 (issue #23749) htt

[issue23749] asyncio missing wrap_socket (starttls)

Alex Grönholm added the comment: So is this going to make it into 3.6...? -- nosy: +Alex Grönholm ___ Python tracker ___ ___ Python-bu

[issue23749] asyncio missing wrap_socket (starttls)

Changes by Christian Heimes : -- assignee: -> christian.heimes components: +SSL nosy: +christian.heimes versions: +Python 3.6, Python 3.7 -Python 3.4 ___ Python tracker ___

[issue23749] asyncio missing wrap_socket (starttls)

Yury Selivanov added the comment: > yuri, did you saw guido added review on your patch? Yes. There are few more issues with the patch that I want to resolve before re-submitting it for another review. Will do it soon. -- ___ Python tracker

[issue23749] asyncio missing wrap_socket (starttls)

박세원 added the comment: https://bugs.python.org/review/23749/#msg1 yuri, did you saw guido added review on your patch? -- nosy: +박세원 ___ Python tracker ___ ___

[issue23749] asyncio missing wrap_socket (starttls)

Yury Selivanov added the comment: I'll create a PR on the GitHub for this. I like the proposed design, and I've implemented an SSL test micro-framework that we can use to test starttls in asyncio. -- ___ Python tracker

[issue23749] asyncio missing wrap_socket (starttls)

Changes by Robert Siemer : -- nosy: +siemer ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python

[issue23749] asyncio missing wrap_socket (starttls)

Yury Selivanov added the comment: Guido, Victor, any thoughts about the (proto-)patch? -- ___ Python tracker ___ ___ Python-bugs-list

[issue23749] asyncio missing wrap_socket (starttls)

Yury Selivanov added the comment: Guido, Victor, Please find attached a first draft of the patch. It's a very early attempt (i.e. I'm not including unit tests/docstrings), and its primary purpose is to gather initial feedback. Some points: 1. As discussed earlier, the primary API point is n

[issue23749] asyncio missing wrap_socket (starttls)

Yury Selivanov added the comment: Here's an example client implementation with writer.start_tls() (taken from my debug code): @asyncio.coroutine def client(addr): reader, writer = yield from asyncio.open_connection( *addr, loop=loop) prin

[issue23749] asyncio missing wrap_socket (starttls)

Elizabeth Myers added the comment: > Therefore, the most convenient place to add the new API are *transports*. I had an inkling this was the case, but I didn't know how to go about the creation of a new protocol and transport pair. > I'm hesitant to add this API to Transport; it somehow feels

[issue23749] asyncio missing wrap_socket (starttls)

Changes by Mathieu Sornay : -- nosy: +msornay ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.pyth

[issue23749] asyncio missing wrap_socket (starttls)

Changes by Antoine Pitrou : -- nosy: -pitrou ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.pyth

[issue23749] asyncio missing wrap_socket (starttls)

Changes by Alex Grönholm : -- nosy: +alex.gronholm ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail

[issue23749] asyncio missing wrap_socket (starttls)

Guido van Rossum added the comment: OK, got it. SGTM. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https:

[issue23749] asyncio missing wrap_socket (starttls)

Yury Selivanov added the comment: > Why does the start_tls() function need to know the internal structure of the > Transport? If start_tls() is added to the loop, it will (likely) have the following signature: loop.start_tls(transport) then I'd want it to check if the `transport` is on

[issue23749] asyncio missing wrap_socket (starttls)

Guido van Rossum added the comment: Why does the start_tls() function need to know the internal structure of the Transport? I'm hesitant to add this API to Transport; it somehow feels wrong to put such an implementation-specific thing there. E.g. I presume you can't do this for an UDP transpor

[issue23749] asyncio missing wrap_socket (starttls)

Yury Selivanov added the comment: I'm working on porting pypostgresql (pure python postgresql driver) library to use asyncio as its underlying IO machinery. And it appears that PQ3 protocol starts as clear text, and then upgrades to use TLS (if server or connection configured so). I've been

[issue23749] asyncio missing wrap_socket (starttls)

Changes by Elizabeth Myers : -- nosy: -Elizacat ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.p

[issue23749] asyncio missing wrap_socket (starttls)

Changes by STINNER Victor : -- title: asyncio missing wrap_socket -> asyncio missing wrap_socket (starttls) ___ Python tracker ___ ___

[issue23749] asyncio missing wrap_socket

Elizabeth Myers added the comment: After giving this a look over, I think this is over my head. Sorry. -- ___ Python tracker ___ ___ P

[issue23749] asyncio missing wrap_socket

Guido van Rossum added the comment: Looks like Antoine drew the same diagram but quicker. :-) Regarding the waiter arg, you can leave that None if you don't need it. It is intended to give the caller a way to block (using event loop machinery) until the connection is ready. But if your caller

[issue23749] asyncio missing wrap_socket

Antoine Pitrou added the comment: > Reading the source now (just woke up, sorry!), the new protocol thing makes > sense Good :-) > I'm not sure what to do with the waiter argument or how to handle that. I'm not sure. Apparently it's used for create_connection(), so perhaps it's not necessary

[issue23749] asyncio missing wrap_socket

Guido van Rossum added the comment: That sounds like a good plan for the top-level APIs. But I think we may need to think about low-level APIs that handle Transports and Protocols as well. The design I had been thinking of does not do any socket-level manipulation (in fact it doesn't care if

[issue23749] asyncio missing wrap_socket

Antoine Pitrou added the comment: > As for replacing the protocol but keeping the transport, what would > be the semantics of that? The protocol is not really replaced, it's wrapped. Before: SocketTransport <- UserProtocol After: SocketTransport <- (asyncio.sslproto.SSLProtocol <- a

[issue23749] asyncio missing wrap_socket

Elizabeth Myers added the comment: Reading the source now (just woke up, sorry!), the new protocol thing makes sense. I'm not sure what to do with the waiter argument or how to handle that. What I'm really trying to think of here is how to handle copying of state. I guess users will just have

[issue23749] asyncio missing wrap_socket

Elizabeth Myers added the comment: It seems pretty simple to just make a function that returns a new transport, something like "transport = yield from loop.ssl_wrap_transport(transport)". I'm not sure how to handle plaintext data left on the wire, though, unless that's not really a considerati

[issue23749] asyncio missing wrap_socket

Changes by Andrew Svetlov : -- nosy: +asvetlov stage: -> needs patch versions: +Python 3.5 ___ Python tracker ___ ___ Python-bugs-lis

[issue23749] asyncio missing wrap_socket

Antoine Pitrou added the comment: So you need to: - have an API to wrap a clear-text protocol in a SSL protocol (see e.g. BaseProactorEventLoop._make_ssl_transport()... note how there's a waiter argument, what should be done with that?) - be able to replace a protocol with another on the tran

[issue23749] asyncio missing wrap_socket

Guido van Rossum added the comment: We didn't do this originally because the 3.4 SSL module didn't have this functionality (or maybe it was 3.3 that didn't have this) but now that we do have it I'd be very happy if you could implement this! I'm not sure what the right interface is, probably co

[issue23749] asyncio missing wrap_socket

Elizabeth Myers added the comment: For what it's worth, IRC has an optional STARTTLS extension which is implemented by some servers. IMAP and SMTP also have STARTTLS as a fundamental component of their protocols. As gc pointed out, LDAP also supports it. IMO this is a pretty glaring omission.

[issue23749] asyncio missing wrap_socket

Elizabeth Myers added the comment: What needs to be done to make this happen? I can try to implement it. -- nosy: +Elizacat ___ Python tracker ___ ___

[issue23749] asyncio missing wrap_socket

Giovanni Cannata added the comment: Thanks, I will look to the new implementation of ssl in 3.5, and try to adapt it for my project (sldap3). I'd like to help, but I'm not skilled in asynchronous programming so I'm not sure if I succeed. Bye, Giovanni -- _

[issue23749] asyncio missing wrap_socket

STINNER Victor added the comment: Yes, it's not supported yet. It was already requested in this issue: https://code.google.com/p/tulip/issues/detail?id=79 asyncio got a new SSL implementation which makes possible to implement STARTTLS. Are you interested to implement it? -- __

[issue23749] asyncio missing wrap_socket

New submission from Giovanni Cannata: It's not possible to wrap a socket in tls. The StreamWriter object should have an option to start a tls negotiation using the SSLContext of the server. This is needed for protocols the have a "start_tls" feature, for example the ldap protocol. In a non as