[issue41756] Do not always use exceptions to return result from coroutine

2020-09-21 Thread Yury Selivanov


Yury Selivanov  added the comment:

> If you disagree with me, please say why, don't just merge the PR.

Apologies, Mark. I didn't intend to merge something bypassing your opinion; 
just missed your comment between reviewing multiple PRs in a few unrelated 
repos. I'm sorry.

On the actual naming subject, you proposed:

> `PySendResult PyIter_Send(PyObject *obj, PyObject *arg, PyObject **result);`

The problem with using this name is that ideally we should also support 
non-native coroutine and generator implementations (i.e. resolve the "send" 
attribute and call it using Python calling convention). Ideally we should have 
two C APIs: one low-level supporting only native objects and a high level one, 
supporting all kinds of them.

Can we perhaps add both `PyGen_Send()` and `PyCoro_Send()` for now that would 
only accept generators and coroutines respectively? After that we can discuss 
adding a more generic `PyIter_Send`?


> Would you revert the PR, please.

Since this is in 3.10/master that nobody uses right now except us (Python core 
devs), can we just issue a follow up PR to fix whatever is there to fix? I'd 
like to avoid the churn of reverting, and again, I apologize for pushing this a 
bit hastily.  Let me know if you actually want a revert and I'll do that.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-20 Thread Yury Selivanov


Yury Selivanov  added the comment:

Vladimir, could you please submit a PR to update 3.10/whatsnew? Need to mention 
both the new C API and the new perf boost in relevant sections.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

Thanks, Vladimir! Also huge thanks to Mark, Serhiy, and Stefan. If there are 
any nits to fix let's do that in follow up PRs.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-18 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 2b05361bf7cbbd76035206fd9befe87f37489f1e by Vladimir Matveev in 
branch 'master':
bpo-41756: Introduce PyGen_Send C API (GH-22196)
https://github.com/python/cpython/commit/2b05361bf7cbbd76035206fd9befe87f37489f1e


--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Sounds like a good middleground to start: add ``PySendResult `` and 
> `PySendResult PyGen_Send(PyGenObject*, PyObject* PyObject**)` specific to 
> generators and coroutines. 

Yes, it seems that everybody agreed on that. I can give the PR another review 
-- is it ready?

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I would also have preferred a more type specific function, but yeah, as long 
> as the types for which the function would normally be used are special cased 
> early enough in the implementation, it makes no big difference.

Maybe add two API funcs: PyGen_Send (specific to generators & coroutines) and 
PyIter_Send?

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Does it sound correct?

It does, but given the amount of back and forth on this, I'd wait for Serhiy 
and Stefan to confirm if they're OK.

IMO the `PyIter_Send` name is OK (given how generic the implementation will be) 
and returning an enum, while a tad unconventional for the C API is way more 
convenient for the developer.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I guess `PyIter_Send` would imply that this function should work for all 
> inputs (like in https://bugs.python.org/msg377007) which also sounds 
> reasonable.

I like `PyIter_Send` (and why I initially proposed it myself) because it will 
also work with the "send" slot defined on some types if we end up adding one.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-16 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I think it should be specific to generators and coroutines. Calling 
> `PyObject_CallMethodIdOneArg(coro, _send, arg);` and interpreting 
> exceptions to emulate the low level API seems a bit too much.

To add to my point: typically higher-level APIs go under the `PyObject_*` 
namespace, whereas `Py{Type}_*` is more concrete. So I'd make `PyGen_Send` to 
only work with `PyGen` and `PyCoro`.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-16 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Since coroutines inherit the generator protocol more or less, I think 
> "PyGen_Send()" is a more suitable name, better than "PyCoro_Send()".

OK, +1. PyGen_Send it is.

> Also should it be specific to generators/coroutines and accept PyGenObject*

I think it should be specific to generators and coroutines. Calling 
`PyObject_CallMethodIdOneArg(coro, _send, arg);` and interpreting 
exceptions to emulate the low level API seems a bit too much.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-16 Thread Yury Selivanov


Yury Selivanov  added the comment:

> As for returned value, I propose to return -1 in case of error, 1 for yielded 
> value and 0 for returned value (i.e. define PYGEN_RETURN = 0, PYGEN_YIELD = 1 
> and PYGEN_ERROR = -1, but without exposing public names).

Sure, that works.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-16 Thread Yury Selivanov


Yury Selivanov  added the comment:

Mark, Stefan,

I don't want this to be stale so I propose to move with my suggestions:

1. We make the new API public. Mark, if you have objections to that - please 
elaborate with some details. IMO, the corresponding Python API is long public 
and there's no harm in exposing a C version of it. Especially given the fact 
that uvloop, cython, and even asyncio itself will be relying on that API.

2. I propose to name the new API `PyIter_Send`. Motivation: it will work with 
both generators and coroutines and plays nicely with `PyIter_Next`.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-11 Thread Yury Selivanov


Yury Selivanov  added the comment:

@Mark
> Mark Shannon wrote: I don't think [the C-API function] should be public, as a 
> possible further improvement is to stop passing exceptions through a side 
> channel, but in result. Maybe we don't want to do that, but lets' not add to 
> the (already rather large) C-API.

Yeah, we can add it as a "private" function, I'm not entirely opposed to that. 
But... it would be great if Cython and C code could still depend on it and use 
it. And then... why should it be private? The corresponding Python API 
"gen.send()" and "gen.throw()" is public, why can't the C API be public too?

We will not fundamentally change generators (it would be a major backwards 
incompatible change), so committing to a good C API sounds reasonable.

@Mark
> Remember that PyIter_Next() is pretty much the same, though, and it has the 
> standard "return PyObject*" interface. These two would diverge then.

Maybe we should call it `_PyIter_Send()`?  While `.send()` is mostly about 
coroutines, regular generators have the method too, and it would be weird to 
call `_PyCoro_Send` on a generator object.

@Vladimir
> PYGEN_ERROR  | NULL | Regular PyErr_* functions should be used to work 
> with error case

Correct.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-10 Thread Yury Selivanov


Yury Selivanov  added the comment:

Big +1 from me. This is something I always wanted to do myself (since the time 
of PEP 492 & 525 implementations) and I think this is a necessary change. It's 
great that this isn't just a C API UX improvement but also yields a big perf 
improvement.

--
nosy: +Mark.Shannon, lukasz.langa, vstinner

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41733] ContextVar get value is unexpected

2020-09-08 Thread Yury Selivanov


Yury Selivanov  added the comment:

> my expected result is all objects should be different, because the code runs 
> in different context.

Contexts "branch", so if you store something in the context before asyncio.run 
it will store it in the current thread state. And then asyncio.run will pick it 
up.

Also I'd recommend using something else than object IDs - those can be reused 
and can be very confusing when browsing logs -- it might look like it was the 
same object, but in fact it could be to different objects sharing the same ID 
at different points of time.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed
type:  -> behavior

___
Python tracker 
<https://bugs.python.org/issue41733>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41687] sendfile implementation is not compatible with Solaris

2020-09-05 Thread Yury Selivanov

Yury Selivanov  added the comment:


New changeset 8c0be6fd9101746235b63ddfb84106d1e9ca286b by Jakub Kulík in branch 
'master':
bpo-41687: Fix sendfile implementation to work with Solaris (#22040)
https://github.com/python/cpython/commit/8c0be6fd9101746235b63ddfb84106d1e9ca286b


--
nosy: +yselivanov

___
Python tracker 
<https://bugs.python.org/issue41687>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41687] sendfile implementation is not compatible with Solaris

2020-09-05 Thread Yury Selivanov


Change by Yury Selivanov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
type:  -> behavior

___
Python tracker 
<https://bugs.python.org/issue41687>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39010] ProactorEventLoop raises unhandled ConnectionResetError

2020-09-03 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 40e2444c364eede59f193979df5a02ed958f56e9 by Miss Islington (bot) 
in branch '3.8':
bpo-39010: Improve test shutdown (GH-22066) (#22083)
https://github.com/python/cpython/commit/40e2444c364eede59f193979df5a02ed958f56e9


--

___
Python tracker 
<https://bugs.python.org/issue39010>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41696] asyncio.run interacts surprisingly with debug mode

2020-09-03 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset a2680058ee1f43cab282e9f4c2b35fd89464ebb6 by Miss Islington (bot) 
in branch '3.9':
bpo-41696: Fix handling of debug mode in asyncio.run (GH-22069) (#22071)
https://github.com/python/cpython/commit/a2680058ee1f43cab282e9f4c2b35fd89464ebb6


--

___
Python tracker 
<https://bugs.python.org/issue41696>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41696] asyncio.run interacts surprisingly with debug mode

2020-09-03 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 1f5f12737755f246fee83a54c600779ad76934a4 by Miss Islington (bot) 
in branch '3.8':
bpo-41696: Fix handling of debug mode in asyncio.run (GH-22069) (#22072)
https://github.com/python/cpython/commit/1f5f12737755f246fee83a54c600779ad76934a4


--

___
Python tracker 
<https://bugs.python.org/issue41696>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39010] ProactorEventLoop raises unhandled ConnectionResetError

2020-09-03 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 8d68e59f11267ded765d4af6d71a49784a12fad5 by Miss Islington (bot) 
in branch '3.9':
bpo-39010: Improve test shutdown (GH-22066) (#22082)
https://github.com/python/cpython/commit/8d68e59f11267ded765d4af6d71a49784a12fad5


--

___
Python tracker 
<https://bugs.python.org/issue39010>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39010] ProactorEventLoop raises unhandled ConnectionResetError

2020-09-03 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset a986b061a3cd4661eb9f857e2936291f1847bd15 by Miss Islington (bot) 
in branch '3.8':
bpo-39010: Fix errors logged on proactor loop restart (GH-22017) (#22035)
https://github.com/python/cpython/commit/a986b061a3cd4661eb9f857e2936291f1847bd15


--

___
Python tracker 
<https://bugs.python.org/issue39010>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39010] ProactorEventLoop raises unhandled ConnectionResetError

2020-09-03 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 49571c0b0e57e20d85727c738d9a0fe342dd2938 by Miss Islington (bot) 
in branch '3.9':
bpo-39010: Fix errors logged on proactor loop restart (GH-22017) (#22034)
https://github.com/python/cpython/commit/49571c0b0e57e20d85727c738d9a0fe342dd2938


--

___
Python tracker 
<https://bugs.python.org/issue39010>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39010] ProactorEventLoop raises unhandled ConnectionResetError

2020-09-02 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset be435ae2b064dc64f04475bec632862e1dbf605f by Ben Darnell in branch 
'master':
bpo-39010: Improve test shutdown (#22066)
https://github.com/python/cpython/commit/be435ae2b064dc64f04475bec632862e1dbf605f


--

___
Python tracker 
<https://bugs.python.org/issue39010>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41696] asyncio.run interacts surprisingly with debug mode

2020-09-02 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 0770ad948cb6d9f7f6c4002efd83e27c27069808 by Shantanu in branch 
'master':
bpo-41696: Fix handling of debug mode in asyncio.run (#22069)
https://github.com/python/cpython/commit/0770ad948cb6d9f7f6c4002efd83e27c27069808


--

___
Python tracker 
<https://bugs.python.org/issue41696>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41696] asyncio.run interacts surprisingly with debug mode

2020-09-02 Thread Yury Selivanov


Yury Selivanov  added the comment:

Yes, this is a bug. The default for the `debug` parameter should be `None` and 
it should only call `loop.set_debug` if a non-None bool value was explicitly 
passed. Care to submit a PR?

--

___
Python tracker 
<https://bugs.python.org/issue41696>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39010] ProactorEventLoop raises unhandled ConnectionResetError

2020-08-31 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset ea5a6363c3f8cc90b7c0cc573922b10f296073b6 by Ben Darnell in branch 
'master':
bpo-39010: Fix errors logged on proactor loop restart (#22017)
https://github.com/python/cpython/commit/ea5a6363c3f8cc90b7c0cc573922b10f296073b6


--

___
Python tracker 
<https://bugs.python.org/issue39010>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32751] wait_for(future, ...) should wait for the future (even if a timeout occurs)

2020-08-26 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 57b698886b47bb81c782c2ba80a8a72fe66c7aad by Elvis Pranskevichus 
in branch '3.8':
[3.8] bpo-32751: Wait for task cancel in asyncio.wait_for() when timeout <= 0 
(GH-21895) (#21967)
https://github.com/python/cpython/commit/57b698886b47bb81c782c2ba80a8a72fe66c7aad


--

___
Python tracker 
<https://bugs.python.org/issue32751>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37658] In some cases asyncio.wait_for can lead to socket leak.

2020-08-26 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 6e1954cd8286e083e7f8d09516d91b6b15769a4e by Miss Islington (bot) 
in branch '3.8':
bpo-37658: Fix asyncio.wait_for() to respect waited task status (GH-21894) 
(#21965)
https://github.com/python/cpython/commit/6e1954cd8286e083e7f8d09516d91b6b15769a4e


--

___
Python tracker 
<https://bugs.python.org/issue37658>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37658] In some cases asyncio.wait_for can lead to socket leak.

2020-08-26 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset a2118a14627256197bddcf4fcecad4c264c1e39d by Elvis Pranskevichus 
in branch 'master':
bpo-37658: Fix asyncio.wait_for() to respect waited task status (#21894)
https://github.com/python/cpython/commit/a2118a14627256197bddcf4fcecad4c264c1e39d


--

___
Python tracker 
<https://bugs.python.org/issue37658>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32751] wait_for(future, ...) should wait for the future (even if a timeout occurs)

2020-08-26 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset c517fc712105c8e5930cb42baaebdbe37fc3e15f by Elvis Pranskevichus 
in branch 'master':
bpo-32751: Wait for task cancel in asyncio.wait_for() when timeout <= 0 (#21895)
https://github.com/python/cpython/commit/c517fc712105c8e5930cb42baaebdbe37fc3e15f


--

___
Python tracker 
<https://bugs.python.org/issue32751>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41543] contextlib.nullcontext doesn't work with async context managers

2020-08-13 Thread Yury Selivanov


Yury Selivanov  added the comment:

I typically don't like objects that support both `with` and `async with`, but 
in this case I think that's appropriate. Nathaniel, Andrew, what do you think?

--
nosy: +njs

___
Python tracker 
<https://bugs.python.org/issue41543>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41197] Async magic methods in contextlib.closing

2020-08-10 Thread Yury Selivanov


Change by Yury Selivanov :


--
nosy: +asvetlov, njs

___
Python tracker 
<https://bugs.python.org/issue41197>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41197] Async magic methods in contextlib.closing

2020-08-10 Thread Yury Selivanov


Yury Selivanov  added the comment:

I'm OK with adding this, but the close method should be `aclose()`

--

___
Python tracker 
<https://bugs.python.org/issue41197>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41438] TimeoutError behavior changes on async.wait_for from python3.7

2020-07-29 Thread Yury Selivanov


Yury Selivanov  added the comment:

> In python 3.8.4 running this script will print:

> asyncio.TimeoutError happened

This is the expected behavior in 3.8.

> This is a breaking change which I didn't see announced in the changelog.

Yeah, looks like this wasn't mentioned in what's new or elsewhere in the docs. 
:(

Do you want to submit a PR to fix that?

--

___
Python tracker 
<https://bugs.python.org/issue41438>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41317] sock_accept() does not remove server socket reader on cancellation

2020-07-23 Thread Yury Selivanov

Yury Selivanov  added the comment:


New changeset 0dd98c2d00a75efbec19c2ed942923981bc06683 by Alex Grönholm in 
branch 'master':
bpo-41317: Remove reader on cancellation in asyncio.loop.sock_accept() (#21595)
https://github.com/python/cpython/commit/0dd98c2d00a75efbec19c2ed942923981bc06683


--

___
Python tracker 
<https://bugs.python.org/issue41317>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41317] sock_accept() does not remove server socket reader on cancellation

2020-07-22 Thread Yury Selivanov


Yury Selivanov  added the comment:

Alex, do you want to submit a PR?

--

___
Python tracker 
<https://bugs.python.org/issue41317>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41305] Add StreamReader.readinto()

2020-07-16 Thread Yury Selivanov


Yury Selivanov  added the comment:

> By the way if we will eventually combine StreamReader and StreamWriter won't 
> this function (readinto) be useful then?

Yes.  But StreamReader and StreamWriter will stay for the foreseeable future 
for backwards compatibility pretty much frozen in time.

So I'd like to start treating them as legacy APIs now.

--

___
Python tracker 
<https://bugs.python.org/issue41305>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41305] Add StreamReader.readinto()

2020-07-16 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Is it simply combining stream reader and stream writer into a single object 
> and changing the write() function to always wait the write (thus deprecating 
> drain) and that's it?

Pretty much. We might also rename a few APIs here and there, like "close()" 
should become an "async def aclose()"

We also will likely want to define a few ABCs.

Which brings me to the most important point: what we need it not coding it 
(yet), but rather drafting the actual proposal and posting it to 
https://discuss.python.org/c/async-sig/20.  Once a formal proposal is there we 
can proceed with the implementation.

It's a bit of a process to follow, but we have to do it this way.

--
nosy: +aeros, njs

___
Python tracker 
<https://bugs.python.org/issue41305>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41305] Add StreamReader.readinto()

2020-07-15 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Im interested in learning about the new api.

There are two problems with the current API:

1. Reader and writer are separate objects, while they should be one.
2. Writer.write is not a coroutine, although it should be one.

There are other minor nits, but that's the crux of the problem. So we need a 
new streams design + a new set of APIs to work with it (and streams are in many 
places, like in the subprocess APIs).

Trio was going to stabilize their own streaming API and we thought it would be 
great if our new API was compatible with it (not sure if they did stabilize it 
or not).

I was going to lead the project myself (and still am) but dropped the ball and 
we missed 3.9 to do this. If you want to start working on this I'd be glad to 
assist with discussions & reviews.

--

___
Python tracker 
<https://bugs.python.org/issue41305>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41305] Add StreamReader.readinto()

2020-07-15 Thread Yury Selivanov


Yury Selivanov  added the comment:

We don't want to extend StreamReader with new APIs as ultimately we plan to 
deprecate it. A new streams API is needed, perhaps inspired by Trio. Sadly, I'm 
-1 on this one.

--

___
Python tracker 
<https://bugs.python.org/issue41305>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41273] asyncio: proactor read transport: use recv_into instead of recv

2020-07-14 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 568fb0ff4aa641329261cdde20795b0aa9278175 by Tony Solomonik in 
branch 'master':
bpo-41273: asyncio's proactor read transport's better performance by using 
recv_into instead of recv (#21442)
https://github.com/python/cpython/commit/568fb0ff4aa641329261cdde20795b0aa9278175


--
nosy: +yselivanov

___
Python tracker 
<https://bugs.python.org/issue41273>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37179] asyncio loop.start_tls() provide support for TLS in TLS

2020-07-09 Thread Yury Selivanov


Yury Selivanov  added the comment:

> The aiohttp issue says they won't fix this until asyncio supports it. Kinda 
> understand that.

I saw you opened an issue with aiohttp to allow this and they're open to it. I 
hope that will get some movement. It also would be a big test for uvloop's (and 
3.10 asyncio) TLS implementation.

--

___
Python tracker 
<https://bugs.python.org/issue37179>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41247] asyncio.set_running_loop() cache running loop holder

2020-07-09 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 0b6169e391ce6468aad711f08ffb829362293ad5 by Tony Solomonik in 
branch '3.8':
bpo-41247: asyncio.set_running_loop() cache running loop holder (#21406)
https://github.com/python/cpython/commit/0b6169e391ce6468aad711f08ffb829362293ad5


--

___
Python tracker 
<https://bugs.python.org/issue41247>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41202] Allow to provide custom exception handler to asyncio.run()

2020-07-09 Thread Yury Selivanov


Yury Selivanov  added the comment:

> So how about maybe:   

That wouldn't work. You still haven't explained what's wrong with calling `loop 
= asyncio.get_running_loop()` inside `async def main()`. That literally solves 
all problems without the need of us modifying any APIs.

--

___
Python tracker 
<https://bugs.python.org/issue41202>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37179] asyncio loop.start_tls() provide support for TLS in TLS

2020-07-08 Thread Yury Selivanov


Yury Selivanov  added the comment:

Looks like https://github.com/python/cpython/pull/17975 was forgotten and was 
never committed to 3.9. So it's 3.10 now.

Best bet for you is to use uvloop which should support the feature.

--

___
Python tracker 
<https://bugs.python.org/issue37179>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41247] asyncio.set_running_loop() cache running loop holder

2020-07-08 Thread Yury Selivanov


Yury Selivanov  added the comment:

> n python to know if there could be a context switch to get_running_loop while 
> set_running_loop is running.


No, it's protected by the GIL.

Good catch, and merged.

--
nosy: +yselivanov
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue41247>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41244] Change to use str.join() instead of += when concatenating string

2020-07-08 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Changes in bpo-41242 were rejected not because the new code is worse, but 
> because it is not obviously and significantly better that the existing code. 
> Here status quo wins for the same reasons. This rule saves us from endless 
> rewriting the code and allows to focus on important things.

+1.

--

___
Python tracker 
<https://bugs.python.org/issue41244>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22239] asyncio: nested event loop

2020-07-07 Thread Yury Selivanov


Yury Selivanov  added the comment:

> The community is hurting *A LOT* right now because asyncio is intentionally 
> non-compatible with the traditional blocking approach that is not only still 
> prevalent it's one that a lot of us think is *easier* to work with.

Mike, I'm super happy with having you here and I encourage you to propose 
feature requests etc. That said, please don't use arguments like this here. 
Everyone has their own point of view and I, for example, haven't seen the "A 
LOT of community hurt" you're describing. I'm not implying that what you're 
saying is wrong, or that asyncio is perfect; the point is that it's just very 
subjective. The bug tracker is not the medium for these kind of remarks.

> That SQLAlchemy internally is not using this coding style, whether or not 
> that leads to new kinds of bugs, there are new kinds of bugs no matter what 
> kind of code a library uses, I don't think this hurts the user community.

You're free to use whatever approach you want in SQLAlchemy. We're here to 
share our advice and perspective (if we have any) and/or to discuss concrete 
proposals for API improvements or changes.

--

___
Python tracker 
<https://bugs.python.org/issue22239>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22239] asyncio: nested event loop

2020-07-07 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Yeah, writing a trivial "event loop" to drive actually-synchronous code is 
> easy. Try it out:

This is exactly the approach I used in edgedb-python.

> I guess there's technically some overhead, but it's tiny.

Correct, the overhead isn't even detectable in microbenchmarks. In most async 
programs regular function calls dominate awaits by a few factors of magnitude.

> I think dropping 'await' syntax has two major downsides:

For the extra context, in the case of using this approach for something like 
edgedb-python these downsides don't really apply, because we're adapting 
async/await implementation to be sync. The async/await code can handle 
cancellation etc. whereas the sync code only needs to support the general 
protocol parsing flow. FWIW I don't think it would be possible to apply my 
approach to SQLA without a very invasive rewrite, which isn't probably worth it.

> tl;dr: I think switching from async/await -> greenlets would make it much 
> easier to write programs that are 90% correct, and much harder to write 
> programs that are 100% correct. That might be a good tradeoff in some 
> situations, but it's a lot more complicated than it seems.

Yeah, this sums up my opinion on this topic.

Also, having spent a couple of years writing and debugging big and small 
greenlet-heavy code bases I wouldn't want to touch them ever again. Your 
mileage may vary, Mike.

--

___
Python tracker 
<https://bugs.python.org/issue22239>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22239] asyncio: nested event loop

2020-07-06 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I think this is a really critical technique to have so that libraries that 
> mediate between a user-facing facade and TCP based backends no longer have to 
> make a hard choice about if they are to support sync vs. async (or async with 
> an optional sync facade around it).

If this works for such a big and elaborate framework as SQLA, we can definitely 
highlight this as a valid approach and even add a link to a blog post from the 
docs. We'll need to add an asyncio specific FAQ page for that or something 
similar.

Another approach, which would probably be a nonstarter for SQLA, is to use 
async/await for literally everything internally, and provide a tiny synchronous 
facade on top.  Funny thing you don't even need an event loop for that, just 
the basic understanding of how coroutines work internally.  I used this to 
create the edgedb-python package which has both sync and async first-class 
support with one code base.  Sync is even faster there in simple throughput 
benchmarks (as expected).

--

___
Python tracker 
<https://bugs.python.org/issue22239>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41202] Allow to provide custom exception handler to asyncio.run()

2020-07-06 Thread Yury Selivanov


Yury Selivanov  added the comment:

The idiomatic way:


async def main():
  loop = asyncio.get_running_loop()
  loop.set_exception_handler(...)
  # other code

asyncio.run(main())

We don't want to add new arguments to asyncio.run as there would be too many.

--

___
Python tracker 
<https://bugs.python.org/issue41202>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22239] asyncio: nested event loop

2020-07-06 Thread Yury Selivanov


Yury Selivanov  added the comment:

Thanks for posting this, Mike.

> Vague claims of "framework X is faster because it's async" appear, impossible 
> to confirm as it is unknown how much of their performance gains come from the 
> "async" aspect and how much of it is that they happened to rewrite a new 
> framework from scratch in a completely different way (hint: it's the latter).

These kind of claims are not specific to async vs. sync. They are all over the 
place for every two pieces of comparable technologies. While novice users might 
base their technology choice purely on such benchmarks, it's less of an issue 
for startups/tech companies.

That said, I agree with most of your points so far.

> The asyncpg project, one of the few asyncio database drivers that exists, 
> notes in its FAQ "asyncpg uses asynchronous execution model and API, which is 
> fundamentally incompatible with SQLAlchemy" [2], yet we know this is not true 
>  because SQLAlchemy works just fine with gevent and eventlet, with no 
> architectural changes at all.

But it is true. Making asynchronous network requests in asyncio requires 
async/await or using callbacks and it's not possible to do them, say, from 
__getattr__ (you mention this yourself).  This is what that particular comment 
is about, nothing more. Using gevent and eventlet as examples in this 
particular context isn't helping you. Apologies for nitpicking, I know it's not 
the point of this discussion.

> A day later, someone took the same idea and got Flask to work in an asyncio 
> event loop at [5].  The general idea of using greenlet in this way is also 
> present at [6], so I won't be patenting this idea today as oremanj can claim 
> prior art.

Yes, this approach definitely works and I even did that in production myself a 
few years ago with https://github.com/1st1/greenio (it's terribly outdated now).

> The recipe is simple and so far appears to be very effective.

This recipe was one of the reasons why I added `loop.set_task_factory` method 
to the spec, so that it's possible to implement this in an *existing* event 
loop like uvloop. But ultimately asyncio is flexible enough to let users use 
their own event loop which can be compatible with greenlets among other 
improvements.

Ultimately, asyncio will probably never ship with greenlets integration enabled 
by default, but we should definitely make it possible (if there are some 
limitations now).  It doesn't seem to me that nested event loops are needed for 
this, right?

--

___
Python tracker 
<https://bugs.python.org/issue22239>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40967] asyncio.Task.all_tasks() and asyncio.Task.current_task() must be removed in 3.9

2020-06-29 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Optimally, we want to do removals before the beta so that users can prepare 
> accordingly rather than deal with breakage in the final release.

+1 to remove it now.  Up to Lukasz to give us green or red light here, though.

--

___
Python tracker 
<https://bugs.python.org/issue40967>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40802] AbstractEventLoop.shutdown_default_executor breaks backwards compatibility

2020-06-09 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Since asyncio is no longer provisional, should it break backwards 
> compatibility with just a What's New entry?

Adding new APIs to asyncio can't be classified as a backward compatibility 
issue. Otherwise the development of it would stall and nobody would benefit 
from it. uvloop and anyio will be eventually updated to support new APIs, I 
don't see any problem here.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue40802>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40816] Add missed AsyncContextDecorator to contextlib

2020-06-02 Thread Yury Selivanov


Yury Selivanov  added the comment:

This has long been implemented: 
https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager

--
nosy: +yselivanov

___
Python tracker 
<https://bugs.python.org/issue40816>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40844] Alternate ways of running coroutines

2020-06-02 Thread Yury Selivanov


Change by Yury Selivanov :


--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue40844>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40844] Alternate ways of running coroutines

2020-06-02 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I'm suggesting a method on coroutines that runs them without blocking, and 
> will run a callback when it's complete.

And how would that method be implemented? Presumably the event loop would 
execute the coroutine, but that API is already there, and it's called 
create_task.  We will not be adding a builtin method for this.

You can use Task.add_done_callback() to add a callback.

--

___
Python tracker 
<https://bugs.python.org/issue40844>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue30064] BaseSelectorEventLoop.sock_{recv, sendall}() don't remove their callbacks when canceled

2020-05-28 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset dc4eee9e266267498a6b783a0abccc23c06f2b87 by Fantix King in branch 
'master':
bpo-30064: Properly skip unstable loop.sock_connect() racing test (GH-20494)
https://github.com/python/cpython/commit/dc4eee9e266267498a6b783a0abccc23c06f2b87


--

___
Python tracker 
<https://bugs.python.org/issue30064>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue30064] BaseSelectorEventLoop.sock_{recv, sendall}() don't remove their callbacks when canceled

2020-05-27 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 210a137396979d747c2602eeef46c34fc4955448 by Fantix King in branch 
'master':
bpo-30064: Fix asyncio loop.sock_* race condition issue (#20369)
https://github.com/python/cpython/commit/210a137396979d747c2602eeef46c34fc4955448


--

___
Python tracker 
<https://bugs.python.org/issue30064>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40696] "await" hangs in Python3.9.0b1.

2020-05-20 Thread Yury Selivanov


Yury Selivanov  added the comment:

> If someone else agrees, I can create a new issue.

I'd keep this one issue, but really up to you. I don't think I have time in the 
next few days to work on what I proposed but would be happy to brainstorm / 
review PRs.

--

___
Python tracker 
<https://bugs.python.org/issue40696>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40696] "await" hangs in Python3.9.0b1.

2020-05-20 Thread Yury Selivanov


Yury Selivanov  added the comment:

Just a note, __context__ cycles can theoretically be longer than 2 nodes. I've 
encountered cycles like `exc.__context__.__context__.__context__ is exc` a few 
times in my life, typically resulting from some weird third-party libraries.

The only solution is to use a `set()` collection to track already visited 
exceptions.

To make it fast I propose to modify the code to:

1. Do a fast traverse with a regular while loop without tracking (no set())

2. If the number of iterations is longer than 100 and there's still no top 
context found -- go to (3)

3. Do a slower implementation with set()

--

___
Python tracker 
<https://bugs.python.org/issue40696>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40672] asyncio.wait_for: process future result produced during cancelation

2020-05-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

> 2) Add some warning about the value is thrown away (in debug mode) and 
> document it somewhere.

The documentation update is definitely something that needs to be done in 3.9.  
Want to submit a PR?

We can also issue a warning in asyncio debug mode.

I'm really not sure about "1) Return that value from `asyncio.wait_for`.".


> I am a newbie here, so sorry if it is wrong to create such "proposal" issues.

The issue is clear, thanks for submitting it! Keep up the great work.

--
nosy: +chris.jerdonek

___
Python tracker 
<https://bugs.python.org/issue40672>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31033] Add argument to .cancel() of Task and Future

2020-05-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

Elevating to release blocker to make sure it's included. The PR is good.

--

___
Python tracker 
<https://bugs.python.org/issue31033>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31033] Add argument to .cancel() of Task and Future

2020-05-17 Thread Yury Selivanov


Change by Yury Selivanov :


--
priority: normal -> release blocker

___
Python tracker 
<https://bugs.python.org/issue31033>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40607] asyncio.wait_for should reraise future exception even if timeout expires

2020-05-15 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 382a5635bd10c237c3e23e346b21cde27e48d7fa by romasku in branch 
'master':
bpo-40607: Reraise exception during task cancelation in asyncio.wait_for() 
(GH-20054)
https://github.com/python/cpython/commit/382a5635bd10c237c3e23e346b21cde27e48d7fa


--

___
Python tracker 
<https://bugs.python.org/issue40607>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34790] Deprecate passing coroutine objects to asyncio.wait()

2020-05-13 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset de92769d473d1c0955d36da2fc71462621326f00 by jack1142 in branch 
'master':
bpo-34790: add version of removal of explicit passing of coros to 
`asyncio.wait`'s documentation (#20008)
https://github.com/python/cpython/commit/de92769d473d1c0955d36da2fc71462621326f00


--

___
Python tracker 
<https://bugs.python.org/issue34790>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40607] asyncio.wait_for should reraise future exception even if timeout expires

2020-05-13 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I think that in case inner task cancelation fails with some error, 
> asyncio.wait_for should reraise it instead of silently losing it.

+1.

--

___
Python tracker 
<https://bugs.python.org/issue40607>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40257] Improve the use of __doc__ in pydoc

2020-05-13 Thread Yury Selivanov


Change by Yury Selivanov :


--
nosy:  -yselivanov

___
Python tracker 
<https://bugs.python.org/issue40257>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40526] documentation bad on asyncio

2020-05-06 Thread Yury Selivanov


Yury Selivanov  added the comment:

> If so, the main purpose of that example is just to demonstrate basic 
> async/await syntax, and show asyncio.run() for a trivial case to clearly show 
> how it's used at a fundamental level; it's intentional that the more involved 
> examples that demonstrate asynchronous programming are contained in 
> https://docs.python.org/3/library/asyncio-task.html#coroutine. Also, the 
> example is simple and condensed enough that it requires zero additional 
> explanation or context, as should be the case for a simple "hello world" 
> example. Consider the perspective of someone who found the page without 
> having previously seen async/await syntax used.

Yes, exactly.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue40526>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40454] DEBUG kw to asyncio.run overrides DEBUG mode set elsewhere

2020-05-04 Thread Yury Selivanov


Yury Selivanov  added the comment:

Good catch. The function should be fixed to:

  _marker = object()

  def run(coro, *, debug=_marker):
 if debug is not _marker:
loop.set_debug(debug)

--

___
Python tracker 
<https://bugs.python.org/issue40454>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39562] Asynchronous comprehensions don't work in asyncio REPL

2020-05-01 Thread Yury Selivanov


Change by Yury Selivanov :


--
nosy:  -yselivanov

___
Python tracker 
<https://bugs.python.org/issue39562>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

2020-04-30 Thread Yury Selivanov


Yury Selivanov  added the comment:

IMO this is a 3.9 fix.

--

___
Python tracker 
<https://bugs.python.org/issue29587>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40405] asyncio.as_completed documentation misleading

2020-04-27 Thread Yury Selivanov


Yury Selivanov  added the comment:

> @Yury what do you think?

Yeah, the documentation needs to be fixed.

> Maybe "Returns an iterator of awaitables"?

I'd suggest to change to: "Return an iterator of coroutines.  Each coroutine 
allows to wait for the earliest next result from the set of the remaining 
awaitables."

--

___
Python tracker 
<https://bugs.python.org/issue40405>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39965] await is valid in non async functions if PyCF_ALLOW_TOP_LEVEL_AWAIT is set

2020-03-15 Thread Yury Selivanov


Yury Selivanov  added the comment:

Good catch & PR ;) Thanks

--

___
Python tracker 
<https://bugs.python.org/issue39965>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37497] Add inspect.Signature.from_text().

2020-03-06 Thread Yury Selivanov


Yury Selivanov  added the comment:

I'd be fine with `Signature.from_text()`, but not with `Signature` constructor 
/ `signature()` function accepting both callable and string arguments. Overall, 
I think that we ought to have a real need to add this new API, so unless 
there's a good (or any, really) use case I'd say we shouldn't merge this now.

--

___
Python tracker 
<https://bugs.python.org/issue37497>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39776] PyContextVar_Get(): crash due to race condition in updating tstate->id

2020-03-04 Thread Yury Selivanov


Yury Selivanov  added the comment:

Thank you so much, Stefan, for looking into this. Really appreciate the help.

--

___
Python tracker 
<https://bugs.python.org/issue39776>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37497] Add inspect.Signature.from_text().

2020-02-26 Thread Yury Selivanov


Yury Selivanov  added the comment:

What's the actual use case for exposing this functionality?

--

___
Python tracker 
<https://bugs.python.org/issue37497>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39529] Deprecate get_event_loop()

2020-02-25 Thread Yury Selivanov


Yury Selivanov  added the comment:

> For asyncio.Lock (plus other synchronization primitives) and asyncio.Queue, 
> this would be added in https://github.com/python/cpython/pull/18195. 
> Currently waiting on emanu (author of the PR) to finish up some changes, but 
> it's mostly complete. Could I work on adding it to asyncio.Future and other 
> classes in the meantime?

I think the approach should be different:


  # leading underscore is significant:
  loop = asyncio._get_running_loop()  
  if loop is None:
issue_deprecation_warning()
loop = asyncio.get_event_loop()

--

___
Python tracker 
<https://bugs.python.org/issue39529>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39698] asyncio.sleep() does not adhere to time.sleep() behavior for negative numbers

2020-02-25 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Source?

I could not find a good source, sorry. I remember I had a complaint in uvloop 
to support negative timeouts, but I can't trace it. 

That said, I also distinctly remember seeing code (and writing such code 
myself) that performs computation on timeouts and does not care if the end 
value goes below 0.  It might be a weak data point but it's still a valid one.

> IMHO, deprecating and then removing support for negative argument in 
> `asyncio.sleep()` is very much less breaking compared to issues #36921 and 
> #36373 .

Breaking code/APIs always has a price and we always try to have a very good 
explanation "why" we want to bother ourselves and users to break backwards 
compat.  This one is not worth it IMHO.

We have more breaking API changes that are more substantial coming in future 
versions of asyncio. So I try to limit the impact by only breaking what's 
really necessary.

--

___
Python tracker 
<https://bugs.python.org/issue39698>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue30064] BaseSelectorEventLoop.sock_{recv, sendall}() don't remove their callbacks when canceled

2020-02-25 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I very doubt if any sane code is organizing like this test: start delayed 
> reading, cancel it and read again.

Hm, cancellation should work correctly no matter how "sane" or "insane" the 
user code is.

> The worse, neither previous not current sock_read() implementation doesn't 
> prevent the concurrent reading which basically delivers data in an 
> unpredictable order.

But we're not discussing using a socket concurrently -- asyncio explicitly does 
not support that for the sock_ api. 

AFAICT this issue is about consequent cancel operation not working as expected 
in asyncio, no?

--

___
Python tracker 
<https://bugs.python.org/issue30064>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39623] __str__ and __repr__ for asyncio.Task still omit arg values

2020-02-20 Thread Yury Selivanov


Yury Selivanov  added the comment:

> It does seem like a good solution.

Great. I'll close this issue then as the proposed solution is actually not as 
straightforward as it seems. Task names exist specifically to solve this case.

--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue39623>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39660] Contextvars: Optional callbacks on state change

2020-02-20 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Would there be too much overhead if allowing specification of a python 
> function that contextvars calls on context changes?

Potentially yes, especially if we allow more than one context change callback.  
Allowing just one makes the API inflexible (what if you want to use two 
libraries from PyPI that both want to use the callback).  Allowing multiple 
context change callbacks leads to complicated API.

For extra context: context switches occur on every callback invocation in 
asyncio and there can be thousands of them per seconds (or even more). Adding 
any extra code to context switching code will noticeably degrade the 
performance.

In general, I'd suggest patching the C library to make state management 
customizable (like CPython allows you to customize which memory allocator to 
use)

--

___
Python tracker 
<https://bugs.python.org/issue39660>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39623] __str__ and __repr__ for asyncio.Task still omit arg values

2020-02-20 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I agree, but wouldn't you agree that some information is better than no 
> information?

We do agree with that. Making it work in the way that does not disturb people 
when a 10mb bytes string is passed is challenging. We could just cut everything 
after 100 characters, but it's not an ideal solution either.

> But in case the same task is run many times with different arguments, the 
> task's name by itself doesn't provide very useful information...

So make it useful. You can concatenate critical arguments reprs to task names 
or make them informative in other way.  If you're working with a third-party 
library that doesn't use task names consider making a PR.

--

___
Python tracker 
<https://bugs.python.org/issue39623>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39660] Contextvars: Optional callbacks on state change

2020-02-20 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Is there any existing API that can be used to call `lib.set_state` on context 
> changes?

No, but there's C API that you can use to get/set contextvars. If a C library 
is hard coded to use threadlocals I'm afraid there's nothing we can do about it 
except fixing their C code to make state storage pluggable.

What library do you have in mind?

> If not, would it make sense to extend `contexvars` to allow users to 
> configure that `lib.set_state` is called on context change?

Theoretically yes, for debug purposes at least.  But I still fail to see how 
you could use that API even if it existed.

--

___
Python tracker 
<https://bugs.python.org/issue39660>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39698] asyncio.sleep() does not adhere to time.sleep() behavior for negative numbers

2020-02-20 Thread Yury Selivanov


Yury Selivanov  added the comment:

> The ship has sailed, this change breaks a lot of existing code without a 
> strong reason.

Yes.

It's a common thing to compute asyncio.sleep delay and sometimes it goes 
negative. The current behavior is part of our API now.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue39698>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39660] Contextvars: Optional callbacks on state change

2020-02-19 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Unfortunately, if Python is used as a frontend for a native libray (eg 
> accessed via ctypes), and in case that the state of interest is managed in 
> the native library, contextvar API is insufficient.

Please elaborate with a clearer example.

--

___
Python tracker 
<https://bugs.python.org/issue39660>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39483] Proposial add loop parametr to run in asyncio

2020-02-11 Thread Yury Selivanov

Yury Selivanov  added the comment:

Андрей,

Here's how you can fix your example:


import asyncio


class Singleton:
_LOCK = None
_LOOP = None

@classmethod
async def create(cls):
if cls._LOOP is None:
cls._LOOP = asyncio.get_running_loop()
cls._LOCK = asyncio.Lock()
loop = cls._LOOP
if loop is not asyncio.get_running_loop():
raise RuntimeError()

if not hasattr(cls, '_Singleton__instance'):
async with cls._LOCK:
if not hasattr(cls, '_Singleton__instance'):
await asyncio.sleep(1)
cls.__instance = cls()
return cls.__instance

--

___
Python tracker 
<https://bugs.python.org/issue39483>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37088] Add a way to schedule a function to be called from the main thread

2020-02-11 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I'm not strongly against the feature. I first proposed to expose it, but make 
> it private. Almost one year later, the PR was not updated. So I just closed 
> the PR and the issue.

All clear, Victor. Let's keep this closed. The reason I didn't reply is because 
we are no longer need it and this is a very low-level piece of functionality 
that I myself feel uneasy about. It can have potential problems to implement in 
PyPy/other interpreters in a compatible with CPython fashion.

I'll reopen if there's a need for this in asyncio. But even then, we can make 
this a "private" and undocumented function in the sys module (of course that 
would mean that we only use it to issue warnings / add extra debug 
functionality).

--

___
Python tracker 
<https://bugs.python.org/issue37088>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37088] Add a way to schedule a function to be called from the main thread

2020-02-11 Thread Yury Selivanov


Yury Selivanov  added the comment:

> There is not clear rationale to justify the addition of the function

Yeah, with the new threaded watcher being the default we don't need this 
anymore.


> so I reject the feature

NP, here, but, hm, can you unilaterally reject features now? :)

--

___
Python tracker 
<https://bugs.python.org/issue37088>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39529] Deprecate get_event_loop()

2020-02-07 Thread Yury Selivanov


Yury Selivanov  added the comment:

I think we still use get_event_loop() in asyncio code, no? If we do, we should 
start by raising deprecation warnings in those call sites, e.g. if a Future or 
Lock is created outside of a coroutine and no explicit event loop is passed. We 
should do this in 3.9. We can then think about deprecating get_event_loop in 
3.10.

--

___
Python tracker 
<https://bugs.python.org/issue39529>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39257] contextvars.Context.run hangs forever in ProccessPoolExecutor

2020-01-10 Thread Yury Selivanov


Yury Selivanov  added the comment:

> This throws 'cannot pickle Context' 

Yes, this is expected. contextvars are not compatible with multiprocessing.

>  This hangs forever *

This hanging part is weird, and most likely hints at a bug in multiprocessing.

--

___
Python tracker 
<https://bugs.python.org/issue39257>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38973] Shared Memory List Returns 0 Length

2019-12-05 Thread Yury Selivanov


Change by Yury Selivanov :


--
nosy:  -yselivanov

___
Python tracker 
<https://bugs.python.org/issue38973>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38979] ContextVar[str] should return ContextVar class, not None

2019-12-05 Thread Yury Selivanov


Yury Selivanov  added the comment:

> The issue is minor, I suspect nobody wants to derive from ContextVar class.

I don't think that's allowed, actually.

> The generic implementation for __class_getitem__ is returning unmodified self 
> argument. Yuri, is there a reason to behave differently in the case of 
> ContextVar?

No, just an oversight, probably.

--

___
Python tracker 
<https://bugs.python.org/issue38979>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37334] Add a cancel method to asyncio Queues

2019-11-22 Thread Yury Selivanov


Yury Selivanov  added the comment:

> 1. A CancelledError (or maybe`QueueCancelled`?) exception is raised in all 
> producers and consumers ) - this gives a producer a chance to handle the 
> error and do something with the waiting item that could not be `put()`

> 2. Items currently on the queue still get processed in the consumers before 
> the consumers exit.

This (especially 2) sounds quite tricky.

Maybe we should just add a method to Queue to get the list of current 
consumers, of pending consumers, and pending producers?  This way users can 
implement whatever semantics they want (probably :wink:).

--

___
Python tracker 
<https://bugs.python.org/issue37334>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37334] Add a cancel method to asyncio Queues

2019-11-21 Thread Yury Selivanov


Yury Selivanov  added the comment:

This seems like a useful idea. I recommend to write a test implementation and 
play with it.


Andrew:
> I think the proposal makes the queues API more error-prone: concurrent put() 
> and close() produces unpredictable result on get() side.

How? Can you elaborate?


Caleb:
> I'm interested in how this change would affect the pattern of shutting down a 
> queue-processing task.

Agree, this can be useful for that.


Martin:
> Given the reactions I gather "close" is a better name for the method, so I 
> changed it accordingly.

Not sure I like "close" since it will *cancel* all getters and putters & 
discard all items in the queue AND allow further operation on the queue.  The 
latter part is really questionable -- what's the point of losing the data in 
the queue and resuming it?  Seems like a mechanism for writing unreliable code, 
but perhaps you can give us an example where this is necessary.

--

___
Python tracker 
<https://bugs.python.org/issue37334>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37228] UDP sockets created by create_datagram_endpoint() allow by default multiple processes to bind the same port

2019-11-20 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Oh in that case, would you like me to close or modify GH-17311? I didn't 
> think you'd approve of making the more extensive changes all the way back to 
> 3.5.

After reading the comments here I think Antoine's solution makes sense. But... 
let's wait a bit to see what other people say.  My comment is just a single +1; 
maybe Andrew, Nathaniel, or Guido have arguments against it.

--
stage: patch review -> 

___
Python tracker 
<https://bugs.python.org/issue37228>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37228] UDP sockets created by create_datagram_endpoint() allow by default multiple processes to bind the same port

2019-11-20 Thread Yury Selivanov


Yury Selivanov  added the comment:

> My preference for create_datagram_endpoint() would be:

> - make the "reuse_address" parameter a no-op, and raise an error when 
> "reuse_address=True" is passed

> - do that in 3.8 as well

Yeah, I like this prposal; we can apply this to all Python's from 3.5 to 3.8.  
With a proper documentation update this should be OK.

--

___
Python tracker 
<https://bugs.python.org/issue37228>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38856] wait_closed() can raise ConnectionResetError

2019-11-19 Thread Yury Selivanov


New submission from Yury Selivanov :

The exception should probably be just ignored.  Andrew, thoughts?

Here's an example error traceback:

  Traceback (most recent call last):
File "c:\projects\asyncpg\asyncpg\connection.py", line 1227, in _cancel
  await w.wait_closed()
File "C:\Python38\lib\asyncio\streams.py", line 376, in wait_closed
  await self._protocol._get_close_waiter(self)
File "c:\projects\asyncpg\asyncpg\connection.py", line 1202, in _cancel
  await r.read()  # Wait until EOF
File "C:\Python38\lib\asyncio\streams.py", line 694, in read
  block = await self.read(self._limit)
File "C:\Python38\lib\asyncio\streams.py", line 701, in read
  await self._wait_for_data('read')
File "C:\Python38\lib\asyncio\streams.py", line 534, in _wait_for_data
  await self._waiter
File "C:\Python38\lib\asyncio\proactor_events.py", line 280, in 
_loop_reading
  data = fut.result()
File "C:\Python38\lib\asyncio\windows_events.py", line 808, in _poll
  value = callback(transferred, key, ov)
File "C:\Python38\lib\asyncio\windows_events.py", line 457, in 
finish_recv
  raise ConnectionResetError(*exc.args)
  ConnectionResetError: [WinError 64] The specified network name is no 
longer available

--
messages: 356996
nosy: asvetlov, lukasz.langa, yselivanov
priority: release blocker
severity: normal
status: open
title: wait_closed() can raise ConnectionResetError
versions: Python 3.8

___
Python tracker 
<https://bugs.python.org/issue38856>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37228] UDP sockets created by create_datagram_endpoint() allow by default multiple processes to bind the same port

2019-11-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

I'd be -1 on changing the default of an existing method, at least without 
consulting with a wider audience.  We can add a new method to the loop and 
deprecate create_datagram_endpoint.

I suggest to post to python-dev and discuss this before making any decisions.

--
nosy: +njs

___
Python tracker 
<https://bugs.python.org/issue37228>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



  1   2   3   4   5   6   7   8   9   10   >