[issue45996] Worse error from asynccontextmanager in Python 3.10
Dima Tisnek added the comment: I'm fine with guarding both. -- ___ Python tracker <https://bugs.python.org/issue45996> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45996] Worse error from asynccontextmanager in Python 3.10
Dima Tisnek added the comment:
Andrew, I see that you've closed this issue as "fixed".
I'm a little confused by that.
If you mean that 3.10 behaviour is better than 3.9, than perhaps "not a bug /
rejected / wont fix" would make more sense.
Actually I don't agree with Thomas's logic... his argument feels like
consistency for its own sake.
I agree with the intention of the change for the arguments to be released
(https://bugs.python.org/issue30306) but I disagree with the cryptic error.
I'm sure we could have both argument release and readable exception, e.g.:
try:
del self.args, ...
except AttributeError:
raise RuntimeError("anext(): asynchronous generator is already running")
--
___
Python tracker
<https://bugs.python.org/issue45996>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45996] Worse error from asynccontextmanager in Python 3.10
New submission from Dima Tisnek : Consider this illegal code: import logging from asyncio import sleep, gather, run from contextlib import asynccontextmanager @asynccontextmanager async def foo(): await sleep(1) yield async def test(): f = foo() await gather(f.__aenter__(), f.__aenter__()) run(test()) If it's ran with Python 3.9, user gets a sensible error: File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/contextlib.py", line 175, in __aenter__ return await self.gen.__anext__() RuntimeError: anext(): asynchronous generator is already running However, if it's ran with Python 3.10, user gets a cryptic error: File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/contextlib.py", line 197, in __aenter__ del self.args, self.kwds, self.func AttributeError: args Which makes it harder to pinpoint what's wrong when the stack is complex. I've hit this with fastapi/starlette/mangum combo and a custom middleware. -- components: asyncio messages: 407799 nosy: Dima.Tisnek, asvetlov, yselivanov priority: normal severity: normal status: open title: Worse error from asynccontextmanager in Python 3.10 type: behavior versions: Python 3.10, Python 3.11 ___ Python tracker <https://bugs.python.org/issue45996> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41271] Add support for io_uring to cpython
Dima Tisnek added the comment: Would now, a year later, be a good time to consider io_uring? -- nosy: +Dima.Tisnek ___ Python tracker <https://bugs.python.org/issue41271> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39951] Ignore specific errors when closing ssl connections
Dima Tisnek added the comment: Thank you, Christian, for removing 3.7 target, I was not up to date on Python support schedule. Regarding protocol violation, let me explain what I've dug up so far... I am not an expert, please feel free to correct me. TLS up to 1.2 Closing a connection (sending close notifies) closes both directions at once. However, something like a data race is possible: Let's say "our" side sent close notify at t0, which arrives at "their" side at t1, and the packet that "their" sent at "t1" arrives back to "us" at t2. "Their" side can legitimately continue to send data during t0..t1, as it has not received the close notify yet. This data can arrive at "our" side anywhere within t0...t2 period. Current implementation treats this data as protocol violation. I don't believe it should, because the data was legitimate at the time it was sent. It should most likely be discarded though. TLS 1.3 and later Closing a connection only closes one half of the connection, the other half is closed after the round trip. (I'm not clear whether that's always the case, or if that's an option). Current implementation is simply unprepared for half-closed connections (I think) and thus in a way, unprepared for TLS 1.3 -- ___ Python tracker <https://bugs.python.org/issue39951> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39951] Ignore specific errors when closing ssl connections
Dima Tisnek added the comment: Added 3.10 target. -- versions: +Python 3.10 ___ Python tracker <https://bugs.python.org/issue39951> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39951] Ignore specific errors when closing ssl connections
Dima Tisnek added the comment: @fantix alas, no: ~/cpython (asvetlov--new-ssl|✚1) [1] > ./python.exe ~/repro-39951.py Traceback (most recent call last): File "/Users/dima.tisnek/repro-39951.py", line 33, in asyncio.run(test()) File "/Users/dima.tisnek/cpython/Lib/asyncio/runners.py", line 43, in run return loop.run_until_complete(main) File "/Users/dima.tisnek/cpython/Lib/asyncio/base_events.py", line 642, in run_until_complete return future.result() File "/Users/dima.tisnek/repro-39951.py", line 30, in test await w.wait_closed() File "/Users/dima.tisnek/cpython/Lib/asyncio/streams.py", line 359, in wait_closed await self._protocol._get_close_waiter(self) File "/Users/dima.tisnek/cpython/Lib/asyncio/sslproto.py", line 638, in _do_shutdown self._sslobj.unwrap() File "/Users/dima.tisnek/cpython/Lib/ssl.py", line 948, in unwrap return self._sslobj.shutdown() ssl.SSLError: [SSL: KRB5_S_INIT] application data after close notify (_ssl.c:2730) -- ___ Python tracker <https://bugs.python.org/issue39951> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36456] task.cancel unbound recursion if task is deadlocked
Change by Dima Tisnek : -- title: task.cancel unbound recursion -> task.cancel unbound recursion if task is deadlocked ___ Python tracker <https://bugs.python.org/issue36456> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36456] task.cancel unbound recursion
Dima Tisnek added the comment: Py3.10: tested on v3.10.0a3:8bae2a958e and v3.10.0a4:445f7f54b1 -- versions: +Python 3.10 ___ Python tracker <https://bugs.python.org/issue36456> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42200] Support TLS 1.3 half-closed connections
New submission from Dima Tisnek : (apologies if this was raised before, I couldn't find it in bugs, mailing lists or discourse). (also apologies if I misunderstood something about the protocol) Up to an including TLS 1.2, a single close notify terminates both upstream and downstream; Starting with TLS 1.3 there are separate close notify alerts for upstream and downstream. This means that it's possible to have a TLS connection in half-closed state, e.g. to send "GET / HTTP xx; close notify" and then wait for the server response. Today, that's not possible, at least in asyncio, as evidenced by https://bugs.python.org/issue39951 (I did not check synchronous wrapped sockets) -- assignee: christian.heimes components: SSL messages: 379908 nosy: Dima.Tisnek, christian.heimes priority: normal severity: normal status: open title: Support TLS 1.3 half-closed connections type: enhancement versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue42200> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37355] SSLSocket.read does a GIL round-trip for every 16KB TLS record
Dima Tisnek added the comment: Amazing! Would you consider making a pull request out of your branch? re: `PyErr_HasSignals` is a notable addition to the cpython API, it may need discussion, and (imo) documentation. -- nosy: +Dima.Tisnek ___ Python tracker <https://bugs.python.org/issue37355> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39951] Ignore specific errors when closing ssl connections
Dima Tisnek added the comment: https://bugs.python.org/issue39953 has landed and the errors are now more sensible: (that patch was not backported to 3.7, because it's not a security issue). Python 3.7.8 [SSL: KRB5_S_INIT] application data after close notify Python 3.8.5 [SSL: APPLICATION_DATA_AFTER_CLOSE_NOTIFY] application data after close notify Python 3.9.0 [SSL: APPLICATION_DATA_AFTER_CLOSE_NOTIFY] application data after close notify Python 3.10.0a1 [SSL: APPLICATION_DATA_AFTER_CLOSE_NOTIFY] application data after close notify I guess I shall work on a patch to ignore this specific error when closing the connection. I would appreciate guidance on half-closed state that, it seems, is possible since TLS 1.3 樂 -- ___ Python tracker <https://bugs.python.org/issue39951> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40060] socket.TCP_NOTSENT_LOWAT is missing in official macOS builds
Dima Tisnek added the comment: Indeed, this issue can be closed, when the mac build update is confirmed to be included in a specific upcoming Python version. -- ___ Python tracker <https://bugs.python.org/issue40060> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12930] reindent.py inserts spaces in multiline literals
Dima Tisnek added the comment: Given the trajectory of this bug, would it be easier to remove `reindent.py` from cpython code base entirely? -- ___ Python tracker <https://bugs.python.org/issue12930> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42117] asyncio calls from sync/async, better docs or api support
Dima Tisnek added the comment: Blaze, I think this needs a concrete proposal: either a patch, or at least a formal example how the new API would be used. Note that there's also async-sig mailing list that's less formal, as well as discuss.python.org to bounce ideas off others. -- nosy: +Dima.Tisnek ___ Python tracker <https://bugs.python.org/issue42117> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7946] Convoy effect with I/O bound threads and New GIL
Dima Tisnek added the comment: My 2c as Python user: Back in 2010, I've used multithreading extensively, both for concurrency and performance. Others used multiprocessing or just shelled out. People talked about using **the other** core, or sometimes the other socket on a server. Now in 2020, I'm using asyncio exclusively. Some colleagues occasionally still shell out . None talking about using all cores on a single machine, rather, we'd spin up dozens of identical containers, which are randomly distributed across N machines, and the synchronisation is offloaded to some database (e.g. atomic ops in redis; transactions in sql). In my imagination, I see future Python as single-threaded (from user's point of view, that is without multithreading api), that features speculative out-of-order async task execution (using hardware threads, maybe pinned) that's invisible to the user. -- ___ Python tracker <https://bugs.python.org/issue7946> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34730] aclose() doesn't stop raise StopAsyncIteration / GeneratorExit to __anext__()
Dima Tisnek added the comment: Then perhaps the issue should be closed 樂 -- ___ Python tracker <https://bugs.python.org/issue34730> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34730] aclose() doesn't stop raise StopAsyncIteration / GeneratorExit to __anext__()
Dima Tisnek added the comment: https://github.com/python/cpython/pull/7468 (prohibit asend/etc. reentrance) was merged ~a year ago. Perhaps it's time to restart the original discussion, whether `aclose()` should cancel pending `anext`. -- nosy: +Dima.Tisnek versions: +Python 3.10 -Python 3.6, Python 3.7 ___ Python tracker <https://bugs.python.org/issue34730> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31861] add aiter() and anext() functions to operator module
Dima Tisnek added the comment: Might as well re-target for 3.10 as 3.9 seems feature-complete now. -- versions: +Python 3.10 -Python 3.8 ___ Python tracker <https://bugs.python.org/issue31861> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40590] test_subprocess stuck on Windows, x64
New submission from Dima Tisnek : The windows test got stuck in my PR, and I'm pretty sure my change is not to blame. The test log looks like this: 0:11:10 load avg: 6.42 [421/423] test_importlib passed 0:11:40 load avg: 5.40 running: test_subprocess (58.5 sec), test_multiprocessing_spawn (53.0 sec) 0:12:10 load avg: 5.48 running: test_subprocess (1 min 28 sec), test_multiprocessing_spawn (1 min 23 sec) 0:12:40 load avg: 5.44 running: test_subprocess (1 min 58 sec), test_multiprocessing_spawn (1 min 53 sec) 0:12:57 load avg: 4.58 [422/423] test_multiprocessing_spawn passed (2 min 10 sec) -- running: test_subprocess (2 min 16 sec) 0:13:27 load avg: 2.89 running: test_subprocess (2 min 46 sec) 0:13:57 load avg: 2.11 running: test_subprocess (3 min 16 sec) 0:14:27 load avg: 1.54 running: test_subprocess (3 min 46 sec) ... 5:53:33 load avg: 0.48 running: test_subprocess (5 hour 42 min) 5:54:03 load avg: 0.51 running: test_subprocess (5 hour 43 min) 5:54:33 load avg: 0.31 running: test_subprocess (5 hour 43 min) Terminate batch job (Y/N)? https://github.com/python/cpython/pull/19402/checks?check_run_id=658308339 -- components: Tests messages: 368614 nosy: Dima.Tisnek, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: test_subprocess stuck on Windows, x64 versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue40590> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39685] Python 3.8 regression Socket operation on non-socket
Dima Tisnek added the comment: The code in question is interesting... I don't claim to understand it. A pair of file descriptors is passed wrapped in sockets, and the objects are used here: https://github.com/sshuttle/sshuttle/blob/966fd0c5231d56c4f464752865d96f97bd3c0aac/sshuttle/ssnet.py#L238-L240 * s.setblocking(False) * os.read(s.fileno()) * select.select([s], [s], ...) elsewhere in the file * s.shutdown() elsewhere in the file server.py uses Mux() with a pair of os.stdin/stdout, realistically pty/pipe/file client.py uses Mux() with [an anonymous domain] socket from socketpair() So, on one hand, it is used like a socket, on the other, it's used as a holder for a file-like file descriptor 路♂️ Back in the day, I too wrote (C) code that treated file/tty and socket file descriptors alike. I've since come to realise that's not exactly correct. IIRC doing so was elegant, UNIX-y, but implied platform specifics, i.e. was not cross-platform. I've also done similar tricks with serial.Serial before: instantiate but don't open, set pty fd, use to test code that expects a serial port in memory. Luckily that code was not open-source ☺️. It was a hack, but I don't know if it's cpython's job to support hacks of this kind. My 2c: Personally, I'd rather see the this "fixed" in sshuttle than in cpython. Perhaps someone with authority over the socket module can weight the balance between "a regression because using socket.socket as a container for generic file descriptors used to work before" vs. "least surprise for well-behaved and future code". -- ___ Python tracker <https://bugs.python.org/issue39685> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31122] SSLContext.wrap_socket() throws OSError with errno == 0
Dima Tisnek added the comment: I know Christian is very busy, so what can I do to have this patch reviewed? * it's concise * there's a reproducer -- ___ Python tracker <https://bugs.python.org/issue31122> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40060] socket.TCP_NOTSENT_LOWAT is missing in official macOS builds
Dima Tisnek added the comment: Wow, very curious. Yes, it's very much like `socket.TCP_KEEPCNT` in that respect, though, admittedly I don't have a very old Mac to test this right now. I think there were VMs for that maybe? 樂 I wonder, what failure would be best for a naive code below, a NameError or OSError(errno=42)? sock.setsockopt(socket.SOL_TCP, socket.TCP_NOTSENT_LOWAT, 42) I guess the question is, at what level should the users catch exceptions... After all, we don't delete this constant on Linux, and surely someone somewhere runs a very old kernel... Oddly according to https://github.com/search?l=Python=%22socket.TCP_NOTSENT_LOWAT%22=Code none (in the OSS community) appears to be using this feature yet? The search without `socket.` prefix yields a bunch of vendored mypy pyi's, but no actual code either. And some even work around the constant being optional: https://github.com/python-trio/trio/blob/5b91edb2a860d024ab057e2be55fb34f311bf8ed/trio/socket.py#L170-L178 So, would this be a "not a regression" if none appears to use this constant yet? Or do we take "don't break existing code" so seriously, that in this case too, we ought to assume that there's someone out there who has private code like below which we must not break?: if code := getattr(socket, "TCP_NOTSENT_LOWAT", None): sock.setsockopt(socket.SOL_TCP, code, 42) P.S. If someone wants to take https://github.com/python/cpython/pull/19402 forward, by all means :) Or I can try to hack up delete-at-runtime... -- ___ Python tracker <https://bugs.python.org/issue40060> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40060] socket.TCP_NOTSENT_LOWAT is missing in official macOS builds
Dima Tisnek added the comment: The constant value is the same for macOS and iOS: iphone, watch, tv: ~ > locate netinet/tcp.h | xargs grep LOWAT /Applications/Xcode.app/Contents/Developer/Platforms/AppleTVOS.platform/Developer/SDKs/AppleTVOS.sdk/usr/include/netinet/tcp.h:#define TCP_NOTSENT_LOWAT 0x201 /* Low water mark for TCP unsent data */ /Applications/Xcode.app/Contents/Developer/Platforms/AppleTVSimulator.platform/Developer/SDKs/AppleTVSimulator.sdk/usr/include/netinet/tcp.h:#define TCP_NOTSENT_LOWAT 0x201 /* Low water mark for TCP unsent data */ /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/netinet/tcp.h:#define TCP_NOTSENT_LOWAT 0x201 /* Low water mark for TCP unsent data */ /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/netinet/tcp.h:#define TCP_NOTSENT_LOWAT 0x201 /* Low water mark for TCP unsent data */ /Applications/Xcode.app/Contents/Developer/Platforms/WatchOS.platform/Developer/SDKs/WatchOS.sdk/usr/include/netinet/tcp.h:#define TCP_NOTSENT_LOWAT 0x201 /* Low water mark for TCP unsent data */ /Applications/Xcode.app/Contents/Developer/Platforms/WatchSimulator.platform/Developer/SDKs/WatchSimulator.sdk/usr/include/netinet/tcp.h:#define TCP_NOTSENT_LOWAT 0x201 /* Low water mark for TCP unsent data */ /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk/usr/include/netinet/tcp.h:#define TCP_NOTSENT_LOWAT 0x201 /* Low water mark for TCP unsent data */ /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator.sdk/usr/include/netinet/tcp.h:#define TCP_NOTSENT_LOWAT 0x201 /* Low water mark for TCP unsent data */ /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/netinet/tcp.h:#define TCP_NOTSENT_LOWAT 0x201 /* Low water mark for TCP unsent data */ /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/netinet/tcp.h:#define TCP_NOTSENT_LOWAT 0x201 /* Low water mark for TCP unsent data */ /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/netinet/tcp.h:#define TCP_NOTSENT_LOWAT 0x201 /* Low water mark for TCP unsent data */ /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/netinet/tcp.h:#define TCP_NOTSENT_LOWAT 0x201 /* Low water mark for TCP unsent data */ -- ___ Python tracker <https://bugs.python.org/issue40060> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40060] socket.TCP_NOTSENT_LOWAT is missing in official macOS builds
Change by Dima Tisnek : -- keywords: +patch pull_requests: +18764 stage: -> patch review pull_request: https://github.com/python/cpython/pull/19402 ___ Python tracker <https://bugs.python.org/issue40060> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40060] socket.TCP_NOTSENT_LOWAT is missing in official macOS builds
Dima Tisnek added the comment: Thank you for the explanation, Ronald. `socket.TCP_NOTSENT_LOWAT` is just a constant though, to be passed to `setsockopt`. What do you think of `ifndef ... define ...` work-around, akin to a few other constants in socket module? https://github.com/python/cpython/blob/799d7d61a91eb0ad3256ef9a45a90029cef93b7c/Modules/socketmodule.h#L162-L168 -- ___ Python tracker <https://bugs.python.org/issue40060> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40060] socket.TCP_NOTSENT_LOWAT is missing in official macOS builds
Dima Tisnek added the comment: +macos team, because I can't for the life of me figure out how official builds are made ☹️ In short: my local build has socket.TCP_NOTSENT_LOWAT but the official build does not. -- nosy: +ned.deily, ronaldoussoren ___ Python tracker <https://bugs.python.org/issue40060> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40111] Introspect ssl context: read ALPN and NPN protocols
New submission from Dima Tisnek : It's quite easy to create new or modify existing ssl context: ssl_context = ssl.create_default_context() ssl_context.set_alpn_protocols(["h2"]) I'm writing a library where the context may be passed by the caller (useful if the caller wants to set custom CA path, or client cert auth, share TLS session tickets, etc.). I'd love to be able to check that the context I get has correct ALPN and/or NPN protocols specified. I'd love to be able to do something like this: assert "h2" in ssl_context.alpn_protocols or assert "h2" in ssl_context.get_alpn_protocols() There's sortof precedent for this, I use following code to set and check TLS version flags: ssl_context.options |= ssl.OP_NO_TLSv1 assert ssl.OP_NO_TLSv1 in ssl_context.options -- components: Extension Modules messages: 365300 nosy: Dima.Tisnek priority: normal severity: normal status: open title: Introspect ssl context: read ALPN and NPN protocols versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue40111> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39951] Ignore specific errors when closing ssl connections
Dima Tisnek added the comment: Sorry I was too fast to close this. 39953 is about error codes. This bug is about having an error at all. I believe that the code in question should pass without error, even in the presence of network delays. -- resolution: duplicate -> status: closed -> open ___ Python tracker <https://bugs.python.org/issue39951> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39951] Ignore specific errors when closing ssl connections
Dima Tisnek added the comment: Let's close this in favour of https://bugs.python.org/issue39953 which has a pending pull request https://github.com/python/cpython/pull/19082 -- resolution: -> duplicate stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue39951> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40060] socket.TCP_NOTSENT_LOWAT is missing in official macOS builds
New submission from Dima Tisnek :
Somehow, it turns out that `TCP_NOTSENT_LOWAT` that's available since 3.7.x is
not available in the official macOS builds :
> python3.7
Python 3.7.4 (v3.7.4:e09359112e, Jul 8 2019, 14:54:52)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.TCP_NOTSENT_LOWAT
Traceback (most recent call last):
File "", line 1, in
AttributeError: module 'socket' has no attribute 'TCP_NOTSENT_LOWAT'
> python3.8
Python 3.8.2 (v3.8.2:7b3ab5921f, Feb 24 2020, 17:52:18)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.TCP_NOTSENT_LOWAT
Traceback (most recent call last):
File "", line 1, in
AttributeError: module 'socket' has no attribute 'TCP_NOTSENT_LOWAT'
> python3.9
Python 3.9.0a4 (v3.9.0a4:6e02691f30, Feb 25 2020, 18:14:13)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.TCP_NOTSENT_LOWAT
Traceback (most recent call last):
File "", line 1, in
AttributeError: module 'socket' has no attribute 'TCP_NOTSENT_LOWAT'
And my local build has it :
> ~/cpython/python.exe
Python 3.9.0a4+ (heads/master:be501ca241, Mar 4 2020, 15:16:49)
[Clang 10.0.1 (clang-1001.0.46.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.TCP_NOTSENT_LOWAT
513
So... my guess is official builds are using old SDK or header files. 樂
My system has it e.g. here:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/netinet/tcp.h
230:#define TCP_NOTSENT_LOWAT 0x201 /* Low water mark for TCP unsent
data */
And in fact it's present in every `netinet/tcp.h` on my system:
CommandLineTools 10.14 and 10.5 sdks; MacOSX dev sdk,
{AppleTV,Watch,iPhone}{OS,Simulator} sdks.
--
components: Extension Modules
messages: 364984
nosy: Dima.Tisnek, Mariatta, njs
priority: normal
severity: normal
status: open
title: socket.TCP_NOTSENT_LOWAT is missing in official macOS builds
versions: Python 3.7, Python 3.8, Python 3.9
___
Python tracker
<https://bugs.python.org/issue40060>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39953] Let's update ssl error codes
New submission from Dima Tisnek :
Let's consider ssl error `291` (https://bugs.python.org/issue39951):
It was introduced into openssl 2 years ago:
https://github.com/openssl/openssl/commit/358ffa05cd3a088822c7d06256bc87516d918798
The documentation states:
SSL_R_APPLICATION_DATA_AFTER_CLOSE_NOTIFY:291:\
application data after close notify
The `ssl.h` header file contains:
# define SSL_R_APPLICATION_DATA_AFTER_CLOSE_NOTIFY291
The master branch of openssl contains this definition too:
https://github.com/openssl/openssl/blob/master/include/openssl/sslerr.h
# define SSL_R_APPLICATION_DATA_AFTER_CLOSE_NOTIFY291
But what does Python say?
ssl.SSLError: [SSL: KRB5_S_INIT] application data after close notify
(_ssl.c:2629)
What's KRB5? It supposedly stands for Kerberos5, and it too is seemingly
present in openssl header file:
/usr/local/Cellar/openssl/1.0.2s/include/openssl/ssl.h
2951:# define SSL_R_KRB5_S_INIT291
Moreover, cpython source code contains a fallback, should this value not be
defined:
https://github.com/python/cpython/blob/master/Modules/_ssl_data.h
#ifdef SSL_R_KRB5_S_INIT
{"KRB5_S_INIT", ERR_LIB_SSL, SSL_R_KRB5_S_INIT},
#else
{"KRB5_S_INIT", ERR_LIB_SSL, 291},
#endif
Thus, today, Python reports an error with wrong *label* but correct *text*:
[SSL: KRB5_S_INIT] application data after close notify
The label and text don't match each other, because... well... I guess that's
why we should fix it :)
--
components: Extension Modules
messages: 364074
nosy: Dima.Tisnek
priority: normal
severity: normal
status: open
title: Let's update ssl error codes
versions: Python 3.8, Python 3.9
___
Python tracker
<https://bugs.python.org/issue39953>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39951] Ignore specific errors when closing ssl connections
Dima Tisnek added the comment:
Reproducer:
""" Reproducer for BPO-39951
We send some data over ssl and close the connection.
The server responds after our openssl considers the connection closed-ish and
raises an error.
"""
import asyncio
import ssl
host = "nghttp2.org"
port = 443
ssl_context = ssl.create_default_context()
ssl_context.options |= ssl.OP_NO_TLSv1
ssl_context.options |= ssl.OP_NO_TLSv1_1
ssl_context.set_alpn_protocols(["h2"])
# Captured from an HTTP/2 client
DATA = b'PRI *
HTTP/2.0\r\n\r\nSM\r\n\r\n\x00\x00*\x04\x00\x00\x00\x00\x00\x00\x01\x00\x00\x10\x00\x00\x02\x00\x00\x00\x00\x00\x04\x00\x00\xff\xff\x00\x05\x00\x00@\x00\x00\x08\x00\x00\x00\x00\x00\x03\x00\x10\x00\x00\x00\x06\x00\x00\xff\xff\x00\x00\x04\x08\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x04\x01\x00\x00\x00\x00'
async def test():
r, w = await asyncio.open_connection(host, port, ssl=ssl_context)
info = w.get_extra_info("ssl_object")
assert info, "HTTP/2 server is required"
proto = info.selected_alpn_protocol()
assert proto == "h2", "Failed to negotiate HTTP/2"
w.write(DATA)
w.close()
await w.wait_closed()
asyncio.run(test())
Test on macOS, using cpython builds from python.org:
… > python3.7 repro-bpo-39951.py
Traceback (most recent call last):
File "repro-bpo-39951.py", line 34, in
asyncio.run(test())
File
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/runners.py",
line 43, in run
return loop.run_until_complete(main)
File
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py",
line 579, in run_until_complete
return future.result()
File "repro-bpo-39951.py", line 31, in test
await w.wait_closed()
File
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/streams.py",
line 323, in wait_closed
await self._protocol._closed
File
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/sslproto.py",
line 530, in data_received
ssldata, appdata = self._sslpipe.feed_ssldata(data)
File
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/sslproto.py",
line 207, in feed_ssldata
self._sslobj.unwrap()
File
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line
778, in unwrap
return self._sslobj.shutdown()
ssl.SSLError: [SSL: KRB5_S_INIT] application data after close notify
(_ssl.c:2629)
… > python3.8 repro-bpo-39951.py
Traceback (most recent call last):
File "repro-bpo-39951.py", line 34, in
asyncio.run(test())
File
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/runners.py",
line 43, in run
return loop.run_until_complete(main)
File
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/base_events.py",
line 616, in run_until_complete
return future.result()
File "repro-bpo-39951.py", line 31, in test
await w.wait_closed()
File
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/streams.py",
line 359, in wait_closed
await self._protocol._get_close_waiter(self)
File
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/sslproto.py",
line 529, in data_received
ssldata, appdata = self._sslpipe.feed_ssldata(data)
File
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/sslproto.py",
line 207, in feed_ssldata
self._sslobj.unwrap()
File
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ssl.py", line
948, in unwrap
return self._sslobj.shutdown()
ssl.SSLError: [SSL: KRB5_S_INIT] application data after close notify
(_ssl.c:2730)
… > python3.9 repro-bpo-39951.py
Traceback (most recent call last):
File "/.../repro-bpo-39951.py", line 34, in
asyncio.run(test())
File
"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/runners.py",
line 43, in run
return loop.run_until_complete(main)
File
"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py",
line 642, in run_until_complete
return future.result()
File "/.../repro-bpo-39951.py", line 31, in test
await w.wait_closed()
File
"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/streams.py",
line 359, in wait_closed
await self._protocol._get_close_waiter(self)
File
"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/sslproto.py",
line 529, in data_received
ssldata, appdata = self._sslpipe.feed_ssldata(data)
File
"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/sslproto.py",
line 207
[issue39951] Ignore specific errors when closing ssl connections
New submission from Dima Tisnek : When a connection wrapped in ssl is closed, sometimes the ssl library reports an error, which I believe should be ignored. The error code is `291` and the name of the error is either SSL_R_KRB5_S_INIT (KRB5_S_INIT) or SSL_R_APPLICATION_DATA_AFTER_CLOSE_NOTIFY depending on openssl header file. It's only one code, somehow `ssl.h` (depending on version?) has different symbolic name for the error. TBH, I consider `KRB5_S_INIT` a misnomer, there's no Kerberos here. The explanation for openssl reporting this error is here: https://github.com/openssl/openssl/blob/6d53ad6b5cf726d92860e973d7bc8c1930762086/ssl/record/rec_layer_s3.c#L1657-L1668 > The peer is continuing to send application data, but we have > already sent close_notify. If this was expected we should have > been called via SSL_read() and this would have been handled > above. This situation is easily achieved, because of network delays. Just because we sent "close notify", doesn't mean the other end has received it, and even if it did, there could still be return data in flight. Reproducer is here: https://gist.github.com/dimaqq/087c66dd7b4a85a669a00221dc3792ea -- components: Extension Modules, Library (Lib) messages: 364071 nosy: Dima.Tisnek priority: normal severity: normal status: open title: Ignore specific errors when closing ssl connections versions: Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue39951> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31122] SSLContext.wrap_socket() throws OSError with errno == 0
Dima Tisnek added the comment: If someone can review https://github.com/python/cpython/pull/18772 then pretty-please review -- ___ Python tracker <https://bugs.python.org/issue31122> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31122] SSLContext.wrap_socket() throws OSError with errno == 0
Dima Tisnek added the comment: https://github.com/python/cpython/pull/18772 posted -- ___ Python tracker <https://bugs.python.org/issue31122> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31122] SSLContext.wrap_socket() throws OSError with errno == 0
Change by Dima Tisnek : -- keywords: +patch pull_requests: +18130 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/18772 ___ Python tracker <https://bugs.python.org/issue31122> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31122] SSLContext.wrap_socket() throws OSError with errno == 0
Dima Tisnek added the comment: I've traced it down to here: https://github.com/python/cpython/blob/be501ca2419a91546dea85ef4f36945545458589/Modules/_ssl.c#L791-L798 `err.c` (errno) == 0, no error, and `err.ssl` == 5, SSL_ERROR_SYSCALL, helpfully commented "look at error stack/return value/errno" in openssl/ssl.h I'm a bit suspicious about `s->errorhandler()` which is some old convention (git blame: 8 years ago), commented "checks errno, returns NULL, set a Python exception", but at this point, we know that errno is 0, so why call it? I'm thinking to just change that to SSLEOFError, but I wonder if something else might break? -- ___ Python tracker <https://bugs.python.org/issue31122> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31122] SSLContext.wrap_socket() throws OSError with errno == 0
Dima Tisnek added the comment: Rejoice: https://github.com/dimaqq/bpo-31122 Short, easy to reproduce :) (I've tested on Mac: 3.7, 3.8, 3.9a from python.org, linked against OpenSSL 1.1.1c/d) Funnily enough, Python 2.7 raises an ssl.SSLEOFError instead 路♂️ -- versions: +Python 3.9 ___ Python tracker <https://bugs.python.org/issue31122> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31122] SSLContext.wrap_socket() throws OSError with errno == 0
Dima Tisnek added the comment: I volunteer to test the theory that the connection is closed mid-flight. -- ___ Python tracker <https://bugs.python.org/issue31122> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38857] AsyncMock issue with awaitable return_value/side_effect/wraps
Dima Tisnek added the comment: Thank you for explanation, Jason! I guess that the bug report and the patch were too technical for me to understand I'm happy with the behaviour in Python 3.8.1 and now I know it's going to stay, I'll just change the tests in our code base. -- ___ Python tracker <https://bugs.python.org/issue38857> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38857] AsyncMock issue with awaitable return_value/side_effect/wraps
Dima Tisnek added the comment:
I think this deserves discussion :)
On one hand, it's a welcome change, on another it's kind of a regression.
Up until 3.8, our tests used to look like this:
---
# code under test
async def foo():
return await bar()
# test
async def helper(value):
return value
async def test_foo():
with patch("bar", return_value=helper(42)):
assert await foo() == 42
---
I feel that the default class `patch()` uses for `new` has crept in too quietly
in 3.8.
At the same time, `helper` was only used because there was no `AsyncMock`.
(or at times, a 3rd party library, `asynctest` was used).
So, on one hand, it's a bit of a regression, but on the other, looking ahead, I
would really like `unittest.mock` to do the right thing.
Can we have it both ways? If not, what way is a better way?
--
nosy: +Dima.Tisnek
___
Python tracker
<https://bugs.python.org/issue38857>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38424] typing.Generator shorthand
Dima Tisnek added the comment: Clearly my attention span does not exceed 5 paragraphs, so sorry! -- ___ Python tracker <https://bugs.python.org/issue38424> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38424] typing.Generator shorthand
Dima Tisnek added the comment: Actually yes it is sufficient. I don't know why I didn't think of that! Perhaps some note in the docs could help other lost children of Python :P -- ___ Python tracker <https://bugs.python.org/issue38424> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38424] typing.Generator shorthand
New submission from Dima Tisnek : Currently to annotate a generator, something like `Generator[str, None, None]` is required. Which is a bit confusing and verbose. Can we allow shorthand, like `Generator[str]` for simple cases? I'm not entirely certain what the semantics ought to be... Maybe Generator[t1, t2=Any, t3=None] ? -- components: Library (Lib) messages: 354307 nosy: Dima.Tisnek priority: normal severity: normal status: open title: typing.Generator shorthand type: enhancement ___ Python tracker <https://bugs.python.org/issue38424> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38413] Remove or change "Multithreading" section
Dima Tisnek added the comment: That would be nice; someone needs to figure out what exact sqlite version allows safe passing of connection between threads. Ideally other caveats would be covered (is passing open cursors to another thread allowed?, etc...) -- nosy: +Dima.Tisnek ___ Python tracker <https://bugs.python.org/issue38413> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38388] Pickle protocol v 5 needs to be documented
Change by Dima Tisnek : -- keywords: +patch pull_requests: +16225 stage: -> patch review pull_request: https://github.com/python/cpython/pull/16639 ___ Python tracker <https://bugs.python.org/issue38388> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38388] Pickle protocol v 5 needs to be documented
New submission from Dima Tisnek : Python 3.8 brings new pickle protocol, version 5. It's not documented. ``` (venv) … ~/m/proj> python Python 3.7.4 (v3.7.4:e09359112e, Jul 8 2019, 14:54:52) [Clang 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import pickle >>> pickle.HIGHEST_PROTOCOL 4 >>> ^D (venv) … ~/m/proj> python3.8 Python 3.8.0rc1 (v3.8.0rc1:34214de6ab, Oct 1 2019, 12:56:49) [Clang 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import pickle >>> pickle.HIGHEST_PROTOCOL 5 >>> ^D ``` Yet there's no mention of "protocol version 5" at: https://github.com/python/cpython/blob/master/Doc/library/pickle.rst https://docs.python.org/3.9/library/pickle.html#data-stream-format https://docs.python.org/3.8/library/pickle.html#data-stream-format -- assignee: docs@python components: Documentation messages: 354054 nosy: Dima.Tisnek, docs@python priority: normal severity: normal status: open title: Pickle protocol v 5 needs to be documented versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue38388> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38342] ImportError: cannot import name 'MetadataPathFinder' from 'importlib.metadata'
Change by Dima Tisnek : -- nosy: +Dima.Tisnek ___ Python tracker <https://bugs.python.org/issue38342> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37657] ERROR: grip 4.5.2 requires docopt>=0.4.0, which is not installed.
Dima Tisnek added the comment: I've nuked 3.8/.../site-packages and the problem is gone; confirmed by reinstalling 3.8 from scratch. It must've been something in the my environment, if/when I find what it was, I'll open a bug in the respective package. -- stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue37657> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37657] ERROR: grip 4.5.2 requires docopt>=0.4.0, which is not installed.
Dima Tisnek added the comment: I've just got it again, when running `Install Certificates.command` after installing Python 3.8.0b4. Yes I do have Python 3.7 installed and had earlier 3.8 builds installed. Could it be referenced by `pip`? Or `certifi` itself? I can't find any references... I can reproduce this at will with: `python -E -s -m pip install --upgrade certifi --force` The `-E -s -m pip` is taken from the cert script, and `--force` is added so that pip doesn't short circuit. Could it be that `grip` is installed badly and that even with `-E -s` it's somehow loaded? -- status: pending -> open ___ Python tracker <https://bugs.python.org/issue37657> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37657] ERROR: grip 4.5.2 requires docopt>=0.4.0, which is not installed.
New submission from Dima Tisnek : I have Python-3.7.2 installed. I've just installed Python3.8.0b2 on macOS, and ran "Install Certificates.command". The terminal output contained: ERROR: grip 4.5.2 requires docopt>=0.4.0, which is not installed. (certifi-2019.6.16 got installed correctly) (Re-running the command again doesn't produce same output, because certifi is up to date) -- components: Installation messages: 348316 nosy: Dima.Tisnek priority: normal severity: normal status: open title: ERROR: grip 4.5.2 requires docopt>=0.4.0, which is not installed. versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue37657> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30550] Document order-preserving dictionary output in json
Change by Dima Tisnek : -- nosy: +Dima.Tisnek ___ Python tracker <https://bugs.python.org/issue30550> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37399] XML text behaviour change if there are comments
Dima Tisnek added the comment: Yes that does look suspicious! -- ___ Python tracker <https://bugs.python.org/issue37399> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37399] XML text behaviour change if there are comments
New submission from Dima Tisnek : Example: # mre.py from xml.etree import ElementTree XML = "foobar" a = ElementTree.fromstring(XML) print(list(a.itertext())) # Testing 3.7.3 vs. 3.8.0b1; macOS … ~> python3.7 mre.py ['foobar'] … ~> python3.8 mre.py ['bar'] -- components: Library (Lib) messages: 346493 nosy: Dima.Tisnek priority: normal severity: normal status: open title: XML text behaviour change if there are comments versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue37399> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31122] SSLContext.wrap_socket() throws OSError with errno == 0
Change by Dima Tisnek : -- nosy: +Dima.Tisnek ___ Python tracker <https://bugs.python.org/issue31122> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37172] Odd error awaiting a Future
Dima Tisnek added the comment: Yes, I think that would be sufficient! I don't know about implementation though, as there's some magic in the way Future.__await__ tries to guess how it was called. I hope one day that magic could go away, but I neither understand the origin nor the mechanics of it, so... My gut tells me that fixing the error involves digging into that magic. -- ___ Python tracker <https://bugs.python.org/issue37172> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37172] Odd error awaiting a Future
Dima Tisnek added the comment: I think that if a Future is abused, the following two should return *same* error: async def test(): await asyncio.gather(reader(), writer()) -vs- async def test(): await asyncio.gather(reader(), reader(), writer()) -- ___ Python tracker <https://bugs.python.org/issue37172> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37172] Odd error awaiting a Future
Dima Tisnek added the comment: Hi Terry, Yes, I have a specific suggestion: The error `RuntimeError: await wasn't used with future` is misleading. I'm not sure if changing error text is enough. I think that Future.__await__ should be fixed; I think that `await f` should be idempotent, that the same exception should be raised in single and double await examples. -- ___ Python tracker <https://bugs.python.org/issue37172> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37171] Documentation mismatch contextvars module vs PEP-567
Dima Tisnek added the comment: It seems I have been blind. I'll close this. -- stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue37171> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37172] Odd error awating a Future
New submission from Dima Tisnek : Let's start with correct code: import asyncio async def writer(): await asyncio.sleep(1) g1.set_result(41) async def reader(): await g1 async def test(): global g1 g1 = asyncio.Future() await asyncio.gather(reader(), writer()) asyncio.run(test()) No error, as expected. Now let's mess it up a bit: import asyncio g1 = asyncio.Future() async def writer(): await asyncio.sleep(1) g1.set_result(41) async def reader(): await g1 async def test(): await asyncio.gather(reader(), writer()) asyncio.run(test()) Fails with RuntimeError ... attached to a different loop The error makes sense, although it's sad that I can't create global futures / there was no even loop when Future was creates, it was not a *different* event loop / maybe I wish .run() didn't force a new event loop? A nit (IMO), but I can live with it. Let's mess the code up a bit more: import asyncio g1 = asyncio.Future() async def writer(): await asyncio.sleep(1) g1.set_result(41) async def reader(): await g1 async def test(): await asyncio.gather(reader(), reader(), writer()) asyncio.run(test()) RuntimeError: await wasn't used with future What? That's really confusing! The only difference is that there are now 2 readers running in parallel. The actual exception comes from asyncio.Future.__await__ after a yield. I'm not sure how to fix this... -- components: asyncio messages: 344798 nosy: Dima.Tisnek, asvetlov, yselivanov priority: normal severity: normal status: open title: Odd error awating a Future versions: Python 3.7, Python 3.8 ___ Python tracker <https://bugs.python.org/issue37172> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37171] Documentation mismatch contextvars module vs PEP-567
New submission from Dima Tisnek : PEP-567 states that user "must call Context.run()" while `contextvars` docs don't mention `.run()` `contextvars.Context().run(arg)` exists, but there's no documentation, for example what the required argument is. -- components: asyncio messages: 344795 nosy: Dima.Tisnek, asvetlov, yselivanov priority: normal severity: normal status: open title: Documentation mismatch contextvars module vs PEP-567 versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue37171> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36456] task.cancel unbound recursion
New submission from Dima Tisnek :
Cancelling a deadlocked group of tasks results in MaximumRecursionError
Inspired by
https://stackoverflow.com/questions/55341189/handling-asyncio-deadlocks
Given the following test-asyncio.py:
```
import asyncio
async def test():
async def f():
await g_task
async def g():
await f_task
f_task = asyncio.create_task(f())
g_task = asyncio.create_task(g())
async def release():
await asyncio.sleep(5)
f_task.cancel()
await asyncio.gather(f_task, g_task, release())
asyncio.run(test())
```
Results in:
```
Traceback (most recent call last):
File ".../python3.8/asyncio/runners.py", line 43, in run
return loop.run_until_complete(main)
File ".../python3.8/asyncio/base_events.py", line 589, in run_until_complete
return future.result()
File "test-asyncio.py", line 18, in test
await asyncio.gather(f_task, g_task, release())
File "test-asyncio.py", line 16, in release
f_task.cancel()
RecursionError: maximum recursion depth exceeded while calling a Python object
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "test-asyncio.py", line 20, in
asyncio.run(test())
File ".../python3.8/asyncio/runners.py", line 46, in run
_cancel_all_tasks(loop)
File ".../python3.8/asyncio/runners.py", line 59, in _cancel_all_tasks
task.cancel()
RecursionError: maximum recursion depth exceeded while calling a Python object
Exception in default exception handler
Traceback (most recent call last):
File ".../python3.8/asyncio/base_events.py", line 1644, in
call_exception_handler
self.default_exception_handler(context)
File ".../python3.8/asyncio/base_events.py", line 1615, in
default_exception_handler
value = repr(value)
File ".../python3.8/asyncio/base_tasks.py", line 21, in _task_repr_info
info.insert(3, f'wait_for={task._fut_waiter!r}')
File ".../python3.8/asyncio/base_tasks.py", line 21, in _task_repr_info
info.insert(3, f'wait_for={task._fut_waiter!r}')
File ".../python3.8/asyncio/base_tasks.py", line 21, in _task_repr_info
info.insert(3, f'wait_for={task._fut_waiter!r}')
[Previous line repeated 326 more times]
File ".../python3.8/asyncio/base_tasks.py", line 9, in _task_repr_info
info = base_futures._future_repr_info(task)
File ".../python3.8/asyncio/base_futures.py", line 57, in _future_repr_info
info.append(_format_callbacks(future._callbacks))
File ".../python3.8/asyncio/base_futures.py", line 36, in _format_callbacks
cb = '{}, {}'.format(format_cb(cb[0][0]), format_cb(cb[1][0]))
File ".../python3.8/asyncio/base_futures.py", line 31, in format_cb
return format_helpers._format_callback_source(callback, ())
File ".../python3.8/asyncio/format_helpers.py", line 23, in
_format_callback_source
func_repr = _format_callback(func, args, None)
File ".../python3.8/asyncio/format_helpers.py", line 56, in _format_callback
func_repr += _format_args_and_kwargs(args, kwargs)
File ".../python3.8/asyncio/format_helpers.py", line 41, in
_format_args_and_kwargs
return '({})'.format(', '.join(items))
RecursionError: maximum recursion depth exceeded while calling a Python object
Exception in default exception handler
Traceback (most recent call last):
File ".../python3.8/asyncio/base_events.py", line 1644, in
call_exception_handler
self.default_exception_handler(context)
File ".../python3.8/asyncio/base_events.py", line 1615, in
default_exception_handler
value = repr(value)
File ".../python3.8/asyncio/base_tasks.py", line 21, in _task_repr_info
info.insert(3, f'wait_for={task._fut_waiter!r}')
File ".../python3.8/asyncio/base_tasks.py", line 21, in _task_repr_info
info.insert(3, f'wait_for={task._fut_waiter!r}')
File ".../python3.8/asyncio/base_tasks.py", line 21, in _task_repr_info
info.insert(3, f'wait_for={task._fut_waiter!r}')
[Previous line repeated 326 more times]
File ".../python3.8/asyncio/base_tasks.py", line 9, in _task_repr_info
info = base_futures._future_repr_info(task)
File ".../python3.8/asyncio/base_futures.py", line 57, in _future_repr_info
info.append(_format_callbacks(future._callbacks))
File ".../python3.8/asyncio/base_futures.py", line 36, in _format_callbacks
cb = '{}, {}'.format(format_cb(cb[0][0]), format_cb(cb[1][0]))
File ".../python3.8/asyncio/base_futures.py", line 31, in format_cb
return format_helpers._format_callback_source(callback, ())
File ".../python3.8/asyncio/format_helpers.py", line 23, in
_format_callback_source
func_repr = _format_callback(func, args, None)
File ".../python3.8/a
[issue36340] 3.7.3rc1 Install Certificates fails on macOS
Dima Tisnek added the comment: I've figured out what's going on: When Installer runs, it asks for user's su passwords, does a bunch of stuff, and then starts "Running package scripts". While it's "running scripts", towards the end of that process, with "about one minute remaining", the Finder window pops up. If "Install Certificates.command" is activated at that time, pip fails. However, if user waits for "running scripts" to complete, the pip succeeds. The "race" window is less than half a minute. Below are the examples of site packages listing during the race window: ``` Tue Mar 19 14:20:50 JST 2019 total 8.0K drwxr-xr-x 28 root admin 896 Mar 19 14:20 setuptools/ drwxrwxr-x 7 root admin 224 Mar 19 14:20 ./ drwxr-xr-x 10 root admin 320 Mar 19 14:20 setuptools-40.8.0.dist-info/ drwxr-xr-x 3 root admin 96 Mar 19 14:20 __pycache__/ -rw-r--r-- 1 root admin 126 Mar 19 14:20 easy_install.py drwxrwxr-x 208 root admin 6.5K Mar 13 05:13 ../ -rw-rw-r-- 1 root admin 119 Mar 13 05:13 README.txt Tue Mar 19 14:20:51 JST 2019 total 8.0K drwxr-xr-x 11 root admin 352 Mar 19 14:20 setuptools-40.8.0.dist-info/ drwxr-xr-x 7 root admin 224 Mar 19 14:20 pkg_resources/ drwxrwxr-x 8 root admin 256 Mar 19 14:20 ./ drwxr-xr-x 42 root admin 1.4K Mar 19 14:20 setuptools/ drwxr-xr-x 3 root admin 96 Mar 19 14:20 __pycache__/ -rw-r--r-- 1 root admin 126 Mar 19 14:20 easy_install.py drwxrwxr-x 208 root admin 6.5K Mar 13 05:13 ../ -rw-rw-r-- 1 root admin 119 Mar 13 05:13 README.txt Tue Mar 19 14:20:52 JST 2019 total 8.0K drwxr-xr-x 6 root admin 192 Mar 19 14:20 pip/ drwxrwxr-x 9 root admin 288 Mar 19 14:20 ./ drwxr-xr-x 11 root admin 352 Mar 19 14:20 setuptools-40.8.0.dist-info/ drwxr-xr-x 7 root admin 224 Mar 19 14:20 pkg_resources/ drwxr-xr-x 42 root admin 1.4K Mar 19 14:20 setuptools/ drwxr-xr-x 3 root admin 96 Mar 19 14:20 __pycache__/ -rw-r--r-- 1 root admin 126 Mar 19 14:20 easy_install.py drwxrwxr-x 208 root admin 6.5K Mar 13 05:13 ../ -rw-rw-r-- 1 root admin 119 Mar 13 05:13 README.txt Tue Mar 19 14:20:53 JST 2019 total 8.0K drwxr-xr-x 3 root admin 96 Mar 19 14:20 __pycache__/ drwxr-xr-x 9 root admin 288 Mar 19 14:20 pip-19.0.3.dist-info/ drwxrwxr-x 10 root admin 320 Mar 19 14:20 ./ drwxr-xr-x 7 root admin 224 Mar 19 14:20 pip/ drwxr-xr-x 11 root admin 352 Mar 19 14:20 setuptools-40.8.0.dist-info/ drwxr-xr-x 7 root admin 224 Mar 19 14:20 pkg_resources/ drwxr-xr-x 42 root admin 1.4K Mar 19 14:20 setuptools/ -rw-r--r-- 1 root admin 126 Mar 19 14:20 easy_install.py drwxrwxr-x 208 root admin 6.5K Mar 13 05:13 ../ -rw-rw-r-- 1 root admin 119 Mar 13 05:13 README.txt ``` I suspect that this is not a big deal in practice, because a new user would not know to run the command until instructed in the "installation completed" screen. Meanwhile, "experienced / trigger-happy" users probably know to run the command again :) P.S. I'm also wondering if it matters that installer asks for current user's su password and after a good installation, most content of site-packages is owned by root:admin, while certifi is owned by current user... For example `__pycache__` not being writable by user. -- Added file: https://bugs.python.org/file48218/Screenshot 2019-03-19 at 14.08.31.png ___ Python tracker <https://bugs.python.org/issue36340> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36340] 3.7.3rc1 Install Certificates fails on macOS
Dima Tisnek added the comment: More system info: I've had 3.7.2 installed from official installer prior to this. Before that, I had 3.7.1 installed from official installer. I also have 3.8a2. 3.6.8, 2.7.15 on the system, as well as an odd version from Homebrew. My system site-packages for 3.7 now (after manual fix) contains ceritifi, pip (19.0.3), setuptools, pkg_resources, easy_install.py and README. -- ___ Python tracker <https://bugs.python.org/issue36340> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36340] 3.7.3rc1 Install Certificates fails on macOS
New submission from Dima Tisnek : I've just installed Python 3.7.3rc1 for macOS 10.9 or later from the macOS 64-bit installer. I've clicked the `Install Certificates.command`, which opened a Terminal, ran and failed with: ``` Last login: Mon Mar 18 16:36:21 on ttys010 Welcome to fish, the friendly interactive shell … ~> /Applications/Python\ 3.7/Install\ Certificates.command ; exit; -- pip install --upgrade certifi Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pip/__main__.py", line 16, in from pip._internal import main as _main # isort:skip # noqa File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pip/_internal/__init__.py", line 19, in from pip._vendor.urllib3.exceptions import DependencyWarning ModuleNotFoundError: No module named 'pip._vendor.urllib3.exceptions' Traceback (most recent call last): File "", line 44, in File "", line 25, in main File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 347, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7', '-E', '-s', '-m', 'pip', 'install', '--upgrade', 'certifi']' returned non-zero exit status 1. [Process completed] ``` I was able to run the same command from my regular shell (iTerm2, fish, $PATH addons, etc.) and that completed just fine and installed certifi==2019.3.9 Either there's just something wrong with my system (what?) or the initial experience for average user is broken (unlikely?)... -- components: Installation messages: 338179 nosy: Dima.Tisnek priority: normal severity: normal status: open title: 3.7.3rc1 Install Certificates fails on macOS versions: Python 3.7 ___ Python tracker <https://bugs.python.org/issue36340> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32528] Change base class for futures.CancelledError
Dima Tisnek added the comment: ping -- nosy: +Dima.Tisnek ___ Python tracker <https://bugs.python.org/issue32528> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31861] add aiter() and anext() functions to operator module
Dima Tisnek added the comment: https://www.python.org/dev/peps/pep-0525/#aiter-and-anext-builtins kinda promised `aiter` and `anext` built-ins. This ticket seems idle. Perhaps it's time for the decider club to either remove that language from PEP-525 or make a plan for aiter/anext? -- nosy: +Dima.Tisnek versions: +Python 3.8 -Python 3.7 ___ Python tracker <https://bugs.python.org/issue31861> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35856] bundled pip syntaxwarning
New submission from Dima Tisnek : It seems that `pip` vendored/bundled with Python3.8 doesn't conform to 3.8 syntax: … ~> /usr/local/bin/python3.8 -m ensurepip /.../tmp.../pip-18.1-py2.py3-none-any.whl/pip/_vendor/requests/status_codes.py:3: SyntaxWarning: invalid escape sequence \o /.../tmp.../pip-18.1-py2.py3-none-any.whl/pip/_vendor/requests/status_codes.py:3: SyntaxWarning: invalid escape sequence \o Looking in links: /.../tmp... Requirement already satisfied: setuptools in /usr/local/lib/python3.8/site-packages (40.6.2) Requirement already satisfied: pip in /usr/local/lib/python3.8/site-packages (18.1) Python 3.8.0a0 (bafa8487f77fa076de3a06755399daf81cb75598) built from source -- components: Library (Lib) messages: 334542 nosy: Dima.Tisnek priority: normal severity: normal status: open title: bundled pip syntaxwarning type: behavior versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue35856> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26467] Add async magic method support to unittest.mock.Mock
Dima Tisnek added the comment:
Perhaps it's possible to vendor asynctest mock in the same vein as `mock` found
it's way into unittest?
The real power of `asynctest` is in constructs like:
with asynctest.mock.patch("module.Class", autospec=True):
...
Where mock instances know what methods are async.
--
nosy: +Dima.Tisnek
versions: +Python 3.8 -Python 3.6, Python 3.7
___
Python tracker
<https://bugs.python.org/issue26467>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35600] Expose siphash
Dima Tisnek added the comment: Steven, my requirement calls for same hash on multiple machines. Python's hash (for strings) is keyed with a random value. You are correct that `hash(tuple(map(ord, str(something` is stable. In the worst case, I could override `PYTHONHASHSEED` globally. I suppose this relegates my suggestion to "why not" or "because we can" category. -- ___ Python tracker <https://bugs.python.org/issue35600> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35600] Expose siphash
New submission from Dima Tisnek : Just recently, i found rolling my own simple hash for strings. (task was to distribute tests across executors, stably across invocations, no external input, no security) In the old days I'd just `hash(some_variable)` but of course now I cannot. `hashlib.sha*` seemed too complex and I ended up with something like `sum(map(ord, str(some_variable)))`. How much easier this would be is `siphash` implementation that cpython uses internally was available to me! -- components: Extension Modules messages: 332633 nosy: Dima.Tisnek priority: normal severity: normal status: open title: Expose siphash type: enhancement versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue35600> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34271] Please support logging of SSL master secret by env variable SSLKEYLOGFILE
Dima Tisnek added the comment: Perhaps https://stackoverflow.com/questions/42332792/chrome-not-firefox-are-not-dumping-to-sslkeylogfile-variable is outdated, but it suggests that: in firefox, this feature os not on by default in chrome, this feature is not available I would be vary of "too much magic"... Though I'd use this in development, I feel that's a bit risky for desktop apps, production, etc... -- nosy: +Dima.Tisnek ___ Python tracker <https://bugs.python.org/issue34271> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29406] asyncio SSL contexts leak sockets after calling close with certain Apache servers
Dima Tisnek added the comment: ping... -- nosy: +Dima.Tisnek ___ Python tracker <https://bugs.python.org/issue29406> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35415] fileno argument to socket.socket is not validated
New submission from Dima Tisnek : socket.socket gained a fileno= kwarg the value of which is not checked if address family and socket type are both provided. For example, following is accepted: >>> socket.socket(socket.AF_INET, socket.SOCK_STREAM, fileno=-1234) >>> socket.socket(socket.AF_INET, socket.SOCK_STREAM, fileno=1234) >>> socket.socket(socket.AF_INET, socket.SOCK_STREAM, fileno=0.999) Resulting in a socket object that will fail at runtime. One of the implications is that it's possible to "steal" file descriptor, i.e. create a socket for an fd that doesn't exist; then some other function/thread happens to create e.g. socket with this specific fd, which can be "unexpectedly" used (or closed or modified, e.g. non-blocking changed) through the first socket object. Additionally if the shorthand is used, the exception raised in these cases has odd text, at least it was misleading for me. >>> socket.socket(fileno=get_wrong_fd_from_somewhere()) [snip] OSError: [Errno 9] Bad file descriptor: 'family' I thought that I had a bug whereby a string was passed in instead of an int fd; Ultimately I had to look in cpython source code to understand what the "family" meant. I volunteer to submit a patch! -- messages: 331096 nosy: Dima.Tisnek priority: normal severity: normal status: open title: fileno argument to socket.socket is not validated versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue35415> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5950] Make zipimport work with zipfile containing comments
Dima Tisnek added the comment: Very glad to hear! Let's document what Python version(s) are "fixed". Perhaps this issue deserves a test case in issue25711. -- ___ Python tracker <https://bugs.python.org/issue5950> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31087] asyncio.create_subprocess_* do not honor `encoding`
Dima Tisnek <dim...@gmail.com> added the comment: I'd love to see universal_newlines=True in asyncio.subprocess. -- nosy: +Dima.Tisnek ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue31087> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30861] StreamReader does not return reamaing and ready data buffer before raise the Exeption
Dima Tisnek added the comment: It seems Guido sets a higher bar on the proposed change. @pfreixes, if you can show that this change is needed to implement "TCP half-closed" state (i.e. when remote calls shutdown(SHUT_WR) after it's done sending data but continues to recv(), then local is expected to read out the data, and then confirm reception and issue it's own shutdown(SHUT_WR) so that remote gets the acknowledgement), then there would be no question. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30861> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30861] StreamReader does not return reamaing and ready data buffer before raise the Exeption
Dima Tisnek added the comment: My 2c: Pau's concern seems valid, in a sense that stream should work like TCP. That's what most users would assume -- read out data until the end, only then you can see what the actual error was (socket closed, or timeout or hard error) However, I suspect the devil may be in details -- the patch appears to assume that exception is only set by the underlying source of data (in which case user wants to read out everything up to the exception). I can imagine a couple of use-case where an exception is set "out of band": * to terminate bad reader (e.g. against Slowris attack) * there's a matching StreamWriter (e.g. shared socket) P.S. If I'm wrong here, then bug report and patch must be clearer :) -- nosy: +Dima.Tisnek ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30861> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30773] async generator receives wrong value when shared between coroutines
Dima Tisnek added the comment:
@Yuri, this bug doesn't require `gather`, here's a version with futures and
explicit await's instead. It produces same output:
```
import asyncio
async def generator():
while True:
x = yield 42
print("received", x)
await asyncio.sleep(0.1)
async def user(name, g):
print("sending", name)
await g.asend(name)
async def helper():
g = generator()
await g.asend(None)
u0 = asyncio.ensure_future(user("user-0", g))
u1 = asyncio.ensure_future(user("user-1", g))
u2 = asyncio.ensure_future(user("user-2", g))
await u0
await u1
await u2
if __name__ == "__main__":
asyncio.get_event_loop().run_until_complete(helper())
```
Same with `asyncio.get_event_loop().create_task` as well.
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue30773>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30773] async generator receives wrong value when shared between coroutines
New submission from Dima Tisnek:
MRE
```
import asyncio
async def generator():
while True:
x = yield 42
print("received", x)
await asyncio.sleep(0.1)
async def user(name, g):
print("sending", name)
await g.asend(name)
async def helper():
g = generator()
await g.asend(None)
await asyncio.gather(*(user(f"user-{x}", g) for x in range(3)))
if __name__ == "__main__":
asyncio.get_event_loop().run_until_complete(helper())
```
Produces output:
```
sending user-0
received user-0
sending user-1
sending user-2
received None
received None
```
Expected output (some variance allowed):
```
sending user-0
received user-0
sending user-1
sending user-2
received user-1
received user-2
```
Initial report / discussion:
https://mail.python.org/pipermail/async-sig/2017-June/000293.html
--
components: asyncio
messages: 296931
nosy: Dima.Tisnek, yselivanov
priority: normal
severity: normal
status: open
title: async generator receives wrong value when shared between coroutines
type: behavior
versions: Python 3.6
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue30773>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28689] OpenSSL 1.1.0c test failures
Dima Tisnek added the comment: 1.1.0d is due today. Who wants to test it? :) -- nosy: +Dima.Tisnek ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28689> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24632] Improve documentation about __main__.py
Dima Tisnek added the comment: +1, I too would like to see this documented -- nosy: +Dima.Tisnek ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24632> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27908] del _limbo[self] KeyError
Dima Tisnek added the comment: Your logic is accurate; _started is in fact separate from _limbo. As such taking a lock for "test-then-set" only would suffice. Now when you bring the other primitive under this lock in one place, it would look cleaner if it was also brought in the other. There's one more issue with proposed change: Before the change, if "already started" exception is ever raised for a given Thread object, then it's guaranteed that that object was started successfully. With the change, it is possible that exception is raised, and thread fails to start, leaving the object in initial state. If it were up to me, I would buy this limitation as price of safety. Someone may disagree. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27908> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27908] del _limbo[self] KeyError
Dima Tisnek added the comment: @rooter, if you go this way, you should also `self._started.set()` with lock held, together with removing thread from `_limbo` -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27908> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25596] Use scandir() to speed up the glob module
Dima Tisnek added the comment: @Serhiy please comment the implications / limitations of the fallback on Windows. Is it that scandir cannot handle bytes argument only? If argument is unicode, but response set contains bytes paths, will that work? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25596> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24139] Use sqlite3 extended error codes
Dima Tisnek added the comment: I meant "the patch makes sense in itself". -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24139> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24139] Use sqlite3 extended error codes
Dima Tisnek added the comment: Aviv the patch makes in itself. Are the changes in the other ticket needed to implement new tests? Or is it possible to include tests here? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24139> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27908] del _limbo[self] KeyError
New submission from Dima Tisnek: To reproduce: ``` import threading import time class U(threading.Thread): def run(self): time.sleep(1) if not x.ident: x.start() x = U() for u in [U() for i in range(1)]: u.start() time.sleep(10) ``` Chance to reproduce ~20% in my setup. This script has a data race (check then act on x.ident). I expected it to occasionally hit `RuntimeError: threads can only be started once` Instead, I get: ``` Unhandled exception in thread started by > Traceback (most recent call last): File "/usr/lib64/python3.5/threading.py", line 882, in _bootstrap self._bootstrap_inner() File "/usr/lib64/python3.5/threading.py", line 906, in _bootstrap_inner del _limbo[self] KeyError: <U(Thread-1, started 139798116361984)> ``` -- components: Library (Lib) messages: 274005 nosy: Dima.Tisnek priority: normal severity: normal status: open title: del _limbo[self] KeyError versions: Python 3.6 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27908> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25596] Use scandir() to speed up the glob module
Dima Tisnek added the comment: Who can review Serhiy's patch? P.S. core of the patch seems good to me, but I'm not an expert on stdlib. -- nosy: +Dima.Tisnek ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue25596> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9303] Migrate sqlite3 module to _v2 API to enhance performance
Dima Tisnek added the comment: Can someone review Aviv's patch? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue9303> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26370] shelve filename inconsistent between platforms
New submission from Dima Tisnek:
shelve.open("foo.db") creates "foo.db" on Linux and "foo.db.db" on OSX.
Something to that extent is even documented:
"""d = shelve.open(filename) # open, with (g)dbm filename -- no suffix"""
and
"""As a side-effect, an extension may be added to the filename and more than
one file may be created."""
Still, it's super-quirky, it's almost as if the message was "don't use shelve."
Some ways out:
* ValueError if "." in basename(filaneme) # a hammer
* ValueError if filename.endswith((".db", ".gdbm", ...)) # block only known
extensions
* strip extension if it's known that underlying library is going to add it
* patch underlying library to always use filename verbatim
--
components: Extension Modules
messages: 260362
nosy: Dima.Tisnek
priority: normal
severity: normal
status: open
title: shelve filename inconsistent between platforms
type: behavior
versions: Python 3.5
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26370>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25758] ensurepip/venv broken on Windows if path includes unicode
New submission from Dima Tisnek: One of my students installed Python 3.5 on Windows 10 to default location where user name "Łukasz" contains unicode. Now "-m venv" and "-m ensurepip" do not work: C:\Users\Łukasz>C:\Users\Łukasz\AppData\Local\Programs\Python\Python35-32\python.exe -m venv workshops Error: Command '['C:\\Users\\\u0141ukasz\\workshops\\Scripts\\python.exe', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1 C:\Users\Łukasz>C:\Users\Łukasz\AppData\Local\Programs\Python\Python35-32\python.exe -m ensurepip Traceback (most recent call last): File "C:\Users\\u0141ukasz\AppData\Local\Programs\Python\Python35-32\lib\runpy.py", line 170, in _run_module_as_main "__main__", mod_spec) File "C:\Users\\u0141ukasz\AppData\Local\Programs\Python\Python35-32\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\Users\\u0141ukasz\AppData\Local\Programs\Python\Python35-32\lib\ensurepip\__main__.py", line 4, in ensurepip._main() File "C:\Users\\u0141ukasz\AppData\Local\Programs\Python\Python35-32\lib\ensurepip\__init__.py", line 209, in _main default_pip=args.default_pip, File "C:\Users\\u0141ukasz\AppData\Local\Programs\Python\Python35-32\lib\ensurepip\__init__.py", line 116, in bootstrap _run_pip(args + [p[0] for p in _PROJECTS], additional_paths) File "C:\Users\\u0141ukasz\AppData\Local\Programs\Python\Python35-32\lib\ensurepip\__init__.py", line 40, in _run_pip import pip File "", line 969, in _find_and_load File "", line 954, in _find_and_load_unlocked File "", line 896, in _find_spec File "", line 1136, in find_spec File "", line 1112, in _get_spec File "", line 1093, in _legacy_get_spec File "", line 444, in spec_from_loader File "", line 530, in spec_from_file_location UnicodeEncodeError: 'mbcs' codec can't encode characters in position 0--1: invalid character -- components: Unicode messages: 255534 nosy: Dima.Tisnek, ezio.melotti, haypo priority: normal severity: normal status: open title: ensurepip/venv broken on Windows if path includes unicode versions: Python 3.5 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25758> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25343] Document atomic operations on builtin types
Changes by Dima Tisnek <dim...@gmail.com>: -- assignee: -> docs@python components: +Documentation nosy: +docs@python type: -> enhancement ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25343> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25343] Document atomic operations on builtin types
New submission from Dima Tisnek: Please document what builtin type operations are actually atomic. For example, what set() operations are atomic? (There are some blogs / tutorials online, but information is outdated and not authoritative) -- messages: 252545 nosy: Dima.Tisnek priority: normal severity: normal status: open title: Document atomic operations on builtin types versions: Python 3.6 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25343> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24758] unittest.mock.Mock's new unsafe feature needs a better error message
Changes by Dima Tisnek dim...@gmail.com: -- nosy: +Dima.Tisnek ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24758 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21238] unittest.mock.Mock should not allow you to use non-existent assert methods
Dima Tisnek added the comment: What is this **assret** spelling? I can't a reference to this spelling anywhere else in the codebase, let alone any docs other that this special kwarg. It seems intentional. Was that a joke? Or something I should know? -- nosy: +Dima.Tisnek ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21238 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18987] distutils.utils.get_platform() for 32-bit Python on a 64-bit machine
Dima Tisnek added the comment: beep! -- nosy: +Dima.Tisnek ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18987 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24139] Use sqlite3 extended error codes
New submission from Dima Tisnek: Let's fetch extended error codes from SQLite, information contained is not particularly interesting to the user, but may be invaluable in debugging! http://www.sqlite.org/rescode.html https://sqlite.org/c3ref/extended_result_codes.html http://www.sqlite.org/c3ref/errcode.html Current behaviour: For example, consider that extended error was SQLITE_READONLY_DBMOVED, it would be set in https://github.com/mackyle/sqlite/blob/1caed0ecc62583c4f8a509ff66ae99b6939fe727/src/pager.c#L4836 without explicit detail string. Python will call sqlite3_errcode() which will strip extended information via ` errMask` in https://github.com/mackyle/sqlite/blob/ebb27fe5bd5045d924d99cdd7dec9b7064c24768/src/main.c#L2176 Python will then call sqlite3_errmsg(), and the error message will be according to truncated (simple) error per: https://github.com/mackyle/sqlite/blob/ebb27fe5bd5045d924d99cdd7dec9b7064c24768/src/main.c#L1318 P.S. there are a few cases where sqlite3 internally sets both error code and message, in those cases, current Python module (presumably) reads out the error message correctly. For example https://github.com/mackyle/sqlite/blob/ebb27fe5bd5045d924d99cdd7dec9b7064c24768/src/main.c#L1584 -- components: Extension Modules messages: 242710 nosy: Dima.Tisnek priority: normal severity: normal status: open title: Use sqlite3 extended error codes type: enhancement versions: Python 3.5, Python 3.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24139 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9303] Migrate sqlite3 module to _v2 API to enhance performance
Dima Tisnek added the comment: Is there any hope? Surely sqlite backwards compatibility is not an issue any longer... -- nosy: +Dima.Tisnek ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9303 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
