[issue44011] Borrow asyncio ssl implementation from uvloop

[Andrew Svetlov]

Andrew Svetlov  added the comment:


New changeset 41ddcd3f40f8171a396e57b841a74fcc83884eab by Kumar Aditya in 
branch 'main':
bpo-44011: Document ssl_shutdown_timeout added by revisited asyncio SSL 
implementation  (GH-31597)
https://github.com/python/cpython/commit/41ddcd3f40f8171a396e57b841a74fcc83884eab


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Kumar Aditya]

Change by Kumar Aditya :


--
pull_requests: +29720
pull_request: https://github.com/python/cpython/pull/31597

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Andrew Svetlov]

Change by Andrew Svetlov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Andrew Svetlov]

Change by Andrew Svetlov :


--
priority: critical -> normal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Andrew Svetlov]

Andrew Svetlov  added the comment:

The code had landed.
Need a follow-up PR with documentation update (mention new ssl_shutdown_timeout 
arguments)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Andrew Svetlov]

Andrew Svetlov  added the comment:


New changeset 13c10bfb777483c7b02877aab029345a056b809c by Kumar Aditya in 
branch 'main':
bpo-44011: New asyncio ssl implementation (#31275)
https://github.com/python/cpython/commit/13c10bfb777483c7b02877aab029345a056b809c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Kumar Aditya]

Change by Kumar Aditya :


--
versions: +Python 3.11 -Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Christian Heimes]

Change by Christian Heimes :


--
nosy:  -christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Kumar Aditya]

Kumar Aditya  added the comment:

I created a draft PR by rebasing the old implementation of 3.10 for 3.11 so we 
can investigate the build-bots failure and fix them so this can be committed 
for 3.11.

See https://github.com/python/cpython/pull/31275

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Kumar Aditya]

Change by Kumar Aditya :


--
pull_requests: +29438
pull_request: https://github.com/python/cpython/pull/31275

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Kumar Aditya]

Kumar Aditya  added the comment:

Since it was reverted as it was beta period, Can this be committed again as 
3.11 is in alpha currently?  @asvetlov

--
nosy: +kumaraditya303

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Unfortunately I have reverted 5fb06edbbb769561e245d0fe13002bab50e2ae60 commit 
to unblock the beta release :(

I know that nobody wants this but my responsibilities as a release manager is 
to safeguard the stability of the release and we are too close to the beta 
release to do all the testing we need, giving that many buildbots have been 
broken in a short timespan.

Andrew, we can try to get your PR merge between beta 1 and beta 2 but once we 
have done extensive testing and we know that there will be no impact on the 
buildbots and the CI.

Thank you all for your understanding

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset 7719953b30430b351ba0f153c2b51b16cc68ee36 by Pablo Galindo in 
branch 'master':
bpo-44011: Revert "New asyncio ssl implementation (GH-17975)" (GH-25848)
https://github.com/python/cpython/commit/7719953b30430b351ba0f153c2b51b16cc68ee36


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Specifically this part of both messages:

>> If your change involves some platform-specific behaviour or has a
>> non-trivial amount of C code, make sure you run the buildbots
>> in your Pull Request by using the "test-with-buildbots" label (
>> https://discuss.python.org/t/now-you-can-test-a-pr-with-the-buildbots-before...).
>> Alternatively you could check the buildbots post-merge in the buildbot 
>> server:
>> https://buildbot.python.org/all/#/builders?tags=%2B3.x=%2Bstable
>> This is very important because if problems are detected at the time >> of the
>> release, the release management team may have to revert
>> the changes and therefore those will not be included in Python 3.10.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I have created https://github.com/python/cpython/pull/25848 for the revert, in 
case this is not fixed in the next hours or so.

Given the nature of PR 17975, this should have tested with the buildbots as the 
release team asked in:

https://mail.python.org/archives/list/python-committ...@python.org/thread/SIJQE3BZ6ICCGNJWFR4YR65BQBJJZZAZ/

and

https://mail.python.org/archives/list/python-committ...@python.org/thread/V7V5JHKZCJVE2GTI5NFEP3PNKOLH35VL/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +24531
pull_request: https://github.com/python/cpython/pull/25848

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Christian Heimes]

Christian Heimes  added the comment:

When was https://buildbot.python.org/all/#/builders/464/builds/138 start? The 
build properties tab doesn't have a start timestamp.

Andrew, increase timeout doesn't seem to help. It's looks like the test suite 
is leaking threads on error.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Since commit 
https://github.com/python/cpython/commit/5fb06edbbb769561e245d0fe13002bab50e2ae60
 was merged there are multiple timeouts in several buildbots. Unfortunately if 
this is not fixed by the time I need to do the beta release I may need to 
revert all these commits until all buildbots are stable again.

Could someone investigate those timeouts?

For instance, check:

https://buildbot.python.org/all/#/builders/464/builds/138

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Andrew Svetlov]

Change by Andrew Svetlov :


--
pull_requests: +24529
pull_request: https://github.com/python/cpython/pull/25846

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Andrew Svetlov]

Change by Andrew Svetlov :


--
pull_requests: +24526
pull_request: https://github.com/python/cpython/pull/25842

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Christian Heimes]

Christian Heimes  added the comment:

I have merged my PR to unblock buildbots. Karthikeyan has made suggestions how 
to improve the tests even further. CI also had some issues with OpenSSL 
3.0.0-alpha15. Please run the tests with new OpenSSL version, too. "make 
multissltest" automates download, compilation, local installation, and testing.

--
assignee:  -> asvetlov
priority: release blocker -> critical

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Christian Heimes]

Christian Heimes  added the comment:


New changeset 37ebdf0a866457ce825d0ff6e498a10938895760 by Christian Heimes in 
branch 'master':
bpo-44011: Fix asyncio tests without ssl module (GH-25840)
https://github.com/python/cpython/commit/37ebdf0a866457ce825d0ff6e498a10938895760


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Christian Heimes]

Christian Heimes  added the comment:

PR GH-25840 fixes most issues. Gentoo with X buildbot 
https://buildbot.python.org/all/#builders/465/builds/23 has one failing test.

==
ERROR: test_create_server_ssl_over_ssl (test.test_asyncio.test_ssl.TestSSL)
--
asyncio.exceptions.CancelledError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File 
"/buildbot/buildarea/cpython/pull_request.ware-gentoo-x86.installed/build/target/lib/python3.10/asyncio/tasks.py",
 line 458, in wait_for
fut.result()
asyncio.exceptions.CancelledError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File 
"/buildbot/buildarea/cpython/pull_request.ware-gentoo-x86.installed/build/target/lib/python3.10/test/test_asyncio/test_ssl.py",
 line 1157, in test_create_server_ssl_over_ssl
self.loop.run_until_complete(start_server())
  File 
"/buildbot/buildarea/cpython/pull_request.ware-gentoo-x86.installed/build/target/lib/python3.10/asyncio/base_events.py",
 line 644, in run_until_complete
return future.result()
  File 
"/buildbot/buildarea/cpython/pull_request.ware-gentoo-x86.installed/build/target/lib/python3.10/test/test_asyncio/test_ssl.py",
 line 1150, in start_server
await asyncio.wait_for(asyncio.gather(*tasks), TIMEOUT)
  File 
"/buildbot/buildarea/cpython/pull_request.ware-gentoo-x86.installed/build/target/lib/python3.10/asyncio/tasks.py",
 line 460, in wait_for
raise exceptions.TimeoutError() from exc
asyncio.exceptions.TimeoutError
--

--
nosy:  -xtreak
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

The PR made sslproto a hard dependency that even import asyncio fails on 
non-ssl builds and thus anything that indirectly import asyncio also fails. It 
seems some of the test modules can be skipped. Some parts of the asyncio 
codebase already has checks for ssl and has to be done for new parts. Attached 
is a patch to add more checks but it will be helpful to ensure only relevant 
parts that absolutely require ssl are skipped.

The test_make_socket_transport is slightly tricky since it tries to simulate 
ssl being not present by patching it but mock does import of sslproto which 
will fail since SSLAgainErrors is initialized at module level. Perhaps the test 
can be modified better to only mock if ssl is not present.


diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py
index e54ee309e4..6ccac76dfb 100644
--- a/Lib/asyncio/base_events.py
+++ b/Lib/asyncio/base_events.py
@@ -41,13 +41,14 @@
 from . import exceptions
 from . import futures
 from . import protocols
-from . import sslproto
 from . import staggered
 from . import tasks
 from . import transports
 from . import trsock
 from .log import logger
 
+if ssl is not None:
+from . import sslproto
 
 __all__ = 'BaseEventLoop',
 
diff --git a/Lib/asyncio/proactor_events.py b/Lib/asyncio/proactor_events.py
index 10852afe2b..ac0dc1978c 100644
--- a/Lib/asyncio/proactor_events.py
+++ b/Lib/asyncio/proactor_events.py
@@ -19,11 +19,17 @@
 from . import futures
 from . import exceptions
 from . import protocols
-from . import sslproto
 from . import transports
 from . import trsock
 from .log import logger
 
+try:
+import ssl
+except ImportError:  # pragma: no cover
+ssl = None
+
+if ssl is not None:
+from . import sslproto
 
 def _set_socket_extra(transport, sock):
 transport._extra['socket'] = trsock.TransportSocket(sock)
@@ -826,6 +832,9 @@ def loop(f=None):
  server, addr, conn)
 protocol = protocol_factory()
 if sslcontext is not None:
+if ssl is None:
+raise RuntimeError('Python ssl module is not 
available')
+
 self._make_ssl_transport(
 conn, protocol, sslcontext, server_side=True,
 extra={'peername': addr}, server=server,
diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py
index 63ab15f30f..9bc9a03699 100644
--- a/Lib/asyncio/selector_events.py
+++ b/Lib/asyncio/selector_events.py
@@ -23,11 +23,12 @@
 from . import events
 from . import futures
 from . import protocols
-from . import sslproto
 from . import transports
 from . import trsock
 from .log import logger
 
+if ssl is not None:
+from . import sslproto
 
 def _test_selector_event(selector, fd, event):
 # Test if the selector is monitoring 'event' events
@@ -213,6 +214,9 @@ def _accept_connection(
 protocol = protocol_factory()
 waiter = self.create_future()
 if sslcontext:
+if ssl is None:
+raise RuntimeError('Python ssl module is not available')
+
 transport = self._make_ssl_transport(
 conn, protocol, sslcontext, waiter=waiter,
 server_side=True, extra=extra, server=server,
diff --git a/Lib/test/test_asyncio/test_selector_events.py 
b/Lib/test/test_asyncio/test_selector_events.py
index 349e4f2dca..6aaa7a86be 100644
--- a/Lib/test/test_asyncio/test_selector_events.py
+++ b/Lib/test/test_asyncio/test_selector_events.py
@@ -70,6 +70,7 @@ def test_make_socket_transport(self):
 
 close_transport(transport)
 
+@unittest.skipIf(ssl is None, 'No ssl module')
 @mock.patch('asyncio.selector_events.ssl', None)
 @mock.patch('asyncio.sslproto.ssl', None)
 def test_make_ssl_transport_without_ssl_error(self):
diff --git a/Lib/test/test_asyncio/test_ssl.py 
b/Lib/test/test_asyncio/test_ssl.py
index 38235c63e0..c58346bcab 100644
--- a/Lib/test/test_asyncio/test_ssl.py
+++ b/Lib/test/test_asyncio/test_ssl.py
@@ -1,3 +1,8 @@
+from test.support import import_helper
+
+# Skip tests if we don't have ssl
+import_helper.import_module('ssl')
+
 import asyncio
 import asyncio.sslproto
 import contextlib
diff --git a/Lib/test/test_asyncio/test_sslproto.py 
b/Lib/test/test_asyncio/test_sslproto.py
index 79a81bd8c3..2edbb11b58 100644
--- a/Lib/test/test_asyncio/test_sslproto.py
+++ b/Lib/test/test_asyncio/test_sslproto.py
@@ -11,6 +11,9 @@
 except ImportError:
 ssl = None
 
+# Skip tests if we don't have ssl
+support.import_helper.import_module('ssl')
+
 import asyncio
 from asyncio import log
 from asyncio import protocols

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list 

[issue44011] Borrow asyncio ssl implementation from uvloop

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Christian Heimes]

Change by Christian Heimes :


--
pull_requests: +24524
pull_request: https://github.com/python/cpython/pull/25840

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Christian Heimes]

Christian Heimes  added the comment:

The commit has broken multiple build bots, e.g 
.https://buildbot.python.org/all/#/builders/345/builds/134/steps/5/logs/stdio 
The new code is missing checks for presence of ssl module. It's an optional 
component.

--
nosy: +christian.heimes
priority: normal -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Andrew Svetlov]

Andrew Svetlov  added the comment:


New changeset 5fb06edbbb769561e245d0fe13002bab50e2ae60 by Andrew Svetlov in 
branch 'master':
bpo-44011: New asyncio ssl implementation (#17975)
https://github.com/python/cpython/commit/5fb06edbbb769561e245d0fe13002bab50e2ae60


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Andrew Svetlov]

Change by Andrew Svetlov :


--
keywords: +patch
pull_requests: +24507
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/17975

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44011] Borrow asyncio ssl implementation from uvloop

[Andrew Svetlov]

New submission from Andrew Svetlov :

There is a PR created a long time ago.
Finally, I've ported tests for it also.

The documentation doesn't mention new ssh_shutdown_timeout parameter yet.

The latest changes from https://github.com/MagicStack/uvloop/pull/385 can be 
applied separately.

--
components: asyncio
messages: 392726
nosy: asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: Borrow asyncio ssl implementation from uvloop
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com