[issue45099] asyncio.Task's documentation says that loop arg is removed when it's not

2022-03-31 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I have a feeling that we need a low-level API section that describes 
> *supported* API that is intended to task extenders only (guys who want to 
> provide an alternative task factory for example).

I'm not sure how exposing all private methods spiraled out of this issue. We 
haven't discussed documenting the low-level methods you just exposed Andrew. 
IMO this should be reverted.

--

___
Python tracker 
<https://bugs.python.org/issue45099>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue47167] Allow overriding future-task compliance check in asyncio

2022-03-31 Thread Yury Selivanov


Yury Selivanov  added the comment:

Sorry, I don't like the committed change and I think it should be reverted.

Futures and Tasks have a documented `get_loop()` method which simply should be 
called by Task machinery, without the need to expose (and document) private 
methods.

--
resolution: fixed -> 
status: closed -> open

___
Python tracker 
<https://bugs.python.org/issue47167>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue45997] asyncio.Semaphore waiters deque doesn't work

2022-03-15 Thread Yury Selivanov


Yury Selivanov  added the comment:

Andrew, the same problem exists in asyncio.Queue which is also critical. Here's 
how I fixed it in edgedb code base: 
https://github.com/edgedb/edgedb/blob/08e41341024828df22a01cd690b11fcff00bca5e/edb/server/compiler_pool/queue.py#L51-L74

--

___
Python tracker 
<https://bugs.python.org/issue45997>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46994] Accept explicit contextvars.Context in asyncio create_task() API

2022-03-13 Thread Yury Selivanov


Yury Selivanov  added the comment:

Yeah, +1 to add a parameter. Fwiw it was on my idea list when i was working on 
the pep

--

___
Python tracker 
<https://bugs.python.org/issue46994>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46829] Confusing CancelError message if multiple cancellations are scheduled

2022-03-13 Thread Yury Selivanov


Yury Selivanov  added the comment:

IOW I think that supporting custom messages is a needless complication of our 
API. Given how complex task trees can become with TaskGroups collecting those 
messages and presenting them all to the user isn't trivial, showing just 
first/last defeats the purpose. So i'm in favor of dropping it.

--

___
Python tracker 
<https://bugs.python.org/issue46829>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46829] Confusing CancelError message if multiple cancellations are scheduled

2022-03-13 Thread Yury Selivanov


Yury Selivanov  added the comment:

Andrew asked me for my opinion on the matter --  I think we should get rid of 
the message. Exception messages for "signals" can be easily lost if an 
exception was re-raised. If the reason of why something is being cancelled is 
important (in my experience it never is) it should be recorded elsewhere.

--

___
Python tracker 
<https://bugs.python.org/issue46829>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46844] Context-based TaskGroup for legacy libraries

2022-02-23 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I believe that this approach would allow more control over tasks implicitly 
> spawned by 3rd-party libraries that cannot control.

Please elaborate. I'm not sure what are the benefits of this.

--

___
Python tracker 
<https://bugs.python.org/issue46844>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46771] Add some form of cancel scopes

2022-02-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

> * `async with` vs `with`: if Andrew thinks `async with` is easier to teach 
> and less error-prone,

It has to be 'async with' like most asyncio apis to avoid user confusion.

--

___
Python tracker 
<https://bugs.python.org/issue46771>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46771] Add some form of cancel scopes

2022-02-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

Couple thoughts:

I'm +1 for adding TaskGroup.cancel() method.

I'd be -1 on abusing `Task.cancel()` to signal something with a nonce. Whatever 
problem we are trying to solve here, it should be solvable without resorting to 
hacks like this. It should be trivial to implement simple tracking of whether a 
child task was cancelled by the group or not to decide on how to handle a rogue 
CancelledError.

--

___
Python tracker 
<https://bugs.python.org/issue46771>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46752] Introduce task groups to asyncio and change task cancellation semantics

2022-02-16 Thread Yury Selivanov


Yury Selivanov  added the comment:

> There's one thing that gather() does that TaskGroup doesn't: it gives us the 
> return values from the tasks.

That's easy to do with task groups too:

  async with TaskGroup() as g:
  r1 = g.create_task(coro1())
  r2 = g.create_task(coro2())

  print(r1.result())

  # or
  print(await r2)  # I *think* this should work

--

___
Python tracker 
<https://bugs.python.org/issue46752>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46347] memory leak in PyEval_EvalCodeEx

2022-01-11 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset be578e0c063dad1dbb273f86d5bc77e4e6f14583 by Yury Selivanov in 
branch 'main':
bpo-46347: Yet another fix in the erorr path of PyEval_EvalCodeEx (#30554)
https://github.com/python/cpython/commit/be578e0c063dad1dbb273f86d5bc77e4e6f14583


--

___
Python tracker 
<https://bugs.python.org/issue46347>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46347] memory leak in PyEval_EvalCodeEx

2022-01-11 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 6f9ca53a6ac343a5663cc5c52546acf9a63b605a by Yury Selivanov in 
branch '3.10':
bpo-46347: Fix PyEval_EvalCodeEx to correctly cleanup in error paths (#30553)
https://github.com/python/cpython/commit/6f9ca53a6ac343a5663cc5c52546acf9a63b605a


--

___
Python tracker 
<https://bugs.python.org/issue46347>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46347] memory leak in PyEval_EvalCodeEx

2022-01-11 Thread Yury Selivanov


Change by Yury Selivanov :


--
pull_requests: +28755
pull_request: https://github.com/python/cpython/pull/30554

___
Python tracker 
<https://bugs.python.org/issue46347>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46347] memory leak in PyEval_EvalCodeEx

2022-01-11 Thread Yury Selivanov


Change by Yury Selivanov :


--
pull_requests: +28754
pull_request: https://github.com/python/cpython/pull/30553

___
Python tracker 
<https://bugs.python.org/issue46347>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46347] memory leak in PyEval_EvalCodeEx

2022-01-11 Thread Yury Selivanov


Change by Yury Selivanov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue46347>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46347] memory leak in PyEval_EvalCodeEx

2022-01-11 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 20b5791ce9b47195ce51cfd5acb223b1ca59cdf0 by Yury Selivanov in 
branch 'main':
bpo-46347: Fix PyEval_EvalCodeEx to correctly cleanup in error paths (#30551)
https://github.com/python/cpython/commit/20b5791ce9b47195ce51cfd5acb223b1ca59cdf0


--

___
Python tracker 
<https://bugs.python.org/issue46347>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46347] memory leak in PyEval_EvalCodeEx

2022-01-11 Thread Yury Selivanov


Yury Selivanov  added the comment:

> The function will still leak "kwnames" and "default" if creating the "func" 
> object fails.

Yeah, here's another PR to address that: 
https://github.com/python/cpython/pull/30551

--

___
Python tracker 
<https://bugs.python.org/issue46347>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46347] memory leak in PyEval_EvalCodeEx

2022-01-11 Thread Yury Selivanov


Change by Yury Selivanov :


--
pull_requests: +28752
pull_request: https://github.com/python/cpython/pull/30551

___
Python tracker 
<https://bugs.python.org/issue46347>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46347] memory leak in PyEval_EvalCodeEx

2022-01-11 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 607d8a838f29ad3c4c4e85b39f338dade5f9cafe by Yury Selivanov in 
branch 'main':
bpo-46347: Fix memory leak in PyEval_EvalCodeEx. (#30546)
https://github.com/python/cpython/commit/607d8a838f29ad3c4c4e85b39f338dade5f9cafe


--

___
Python tracker 
<https://bugs.python.org/issue46347>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46347] memory leak in PyEval_EvalCodeEx

2022-01-11 Thread Yury Selivanov


Change by Yury Selivanov :


--
keywords: +patch
pull_requests: +28746
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30546

___
Python tracker 
<https://bugs.python.org/issue46347>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46347] memory leak in PyEval_EvalCodeEx

2022-01-11 Thread Yury Selivanov


Yury Selivanov  added the comment:

This is the questionable place in the code: 
https://github.com/python/cpython/blob/dce642f24418c58e67fa31a686575c980c31dd37/Python/ceval.c#L6131-L6166

See that `kwargs` is allocated but never freed or used.

--

___
Python tracker 
<https://bugs.python.org/issue46347>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46347] memory leak in PyEval_EvalCodeEx

2022-01-11 Thread Yury Selivanov


Yury Selivanov  added the comment:

(this is the context of our current investigation btw: 
https://github.com/MagicStack/asyncpg/issues/874#issuecomment-1009383262)

--

___
Python tracker 
<https://bugs.python.org/issue46347>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46347] memory leak in PyEval_EvalCodeEx

2022-01-11 Thread Yury Selivanov


New submission from Yury Selivanov :

I'm investigating a memory leak in 3.10, and while looking at the offending 
commit I stumbled upon this: in ceval.c:PyEval_EvalCodeEx, there's this 
allocation:

PyObject **kwargs = PyMem_Malloc(sizeof(PyObject *)*kwcount);

The problem is that this isn't ever freed. And `kwargs` isn't used anywhere in 
the function body. It seems to me that this is silently leaking memory.

--
messages: 410329
nosy: Elvis.Pranskevichus, Mark.Shannon, pablogsal, vstinner, yselivanov
priority: normal
severity: normal
status: open
title: memory leak in PyEval_EvalCodeEx

___
Python tracker 
<https://bugs.python.org/issue46347>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue45123] PyAiter_Check & PyObject_GetAiter issues

2021-09-06 Thread Yury Selivanov


Change by Yury Selivanov :


--
keywords: +patch
pull_requests: +26618
pull_request: https://github.com/python/cpython/pull/28194

___
Python tracker 
<https://bugs.python.org/issue45123>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue45123] PyAiter_Check & PyObject_GetAiter issues

2021-09-06 Thread Yury Selivanov


New submission from Yury Selivanov :

Per discussion on python-dev (also see the linked email), PyAiter_Check should 
only check for `__anext__` existence (and not for `__aiter__`) to be consistent 
with `Py_IterCheck`.

While there, I'd like to rename PyAiter_Check to PyAIter_Check and 
PyObject_GetAiter to PyObject_GetAIter (i -> I).  First, we should apply 
CamelCase convention correctly, here "async" and "iter" are separate words; 
second, "Aiter" is marked as invalid spelling by spell checkers in IDEs which 
is annoying.

See 
https://mail.python.org/archives/list/python-...@python.org/message/BRHMOFPEKGQCCKEKEEKGSYDR6NOPMRCC/
 for more details.

--
components: Interpreter Core
messages: 401206
nosy: pablogsal, yselivanov
priority: release blocker
severity: normal
stage: patch review
status: open
title: PyAiter_Check & PyObject_GetAiter issues
type: behavior
versions: Python 3.10, Python 3.11

___
Python tracker 
<https://bugs.python.org/issue45123>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue45088] Coroutines & async generators disagree on the iteration protocol semantics

2021-09-02 Thread Yury Selivanov


Change by Yury Selivanov :


--
nosy: +gvanrossum

___
Python tracker 
<https://bugs.python.org/issue45088>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue45088] Coroutines & async generators disagree on the iteration protocol semantics

2021-09-02 Thread Yury Selivanov


New submission from Yury Selivanov :

See this script:

  https://gist.github.com/1st1/eccc32991dc2798f3fa0b4050ae2461d

Somehow an identity async function alters the behavior of manual iteration 
though the wrapped nested generator.

This is a very subtle bug and I'm not even sure if this is a bug or not. 
Opening the issue so that I don't forget about this and debug sometime later.

--
components: Interpreter Core
messages: 400951
nosy: lukasz.langa, pablogsal, yselivanov
priority: normal
severity: normal
stage: needs patch
status: open
title: Coroutines & async generators disagree on the iteration protocol 
semantics
type: behavior
versions: Python 3.10, Python 3.11

___
Python tracker 
<https://bugs.python.org/issue45088>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22239] asyncio: nested event loop

2021-07-26 Thread Yury Selivanov


Yury Selivanov  added the comment:

> nest-asyncio library (https://github.com/erdewit/nest_asyncio)

Seeing that the community actively wants to have support for nested loops I'm 
slowly changing my opinion on this.

Guido, maybe we should allow nested asyncio loops disabled by default?

--

___
Python tracker 
<https://bugs.python.org/issue22239>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44604] [Enhancement] Asyncio task decorator to provide interface to define async DAGs (similar to dask's delayed interface)

2021-07-13 Thread Yury Selivanov


Yury Selivanov  added the comment:

Yeah, I'm also not convinced this has to be part of asyncio, especially since 
we'll likely have task groups in 3.11.

--

___
Python tracker 
<https://bugs.python.org/issue44604>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38323] asyncio: MultiLoopWatcher has a race condition (test_asyncio: test_close_kill_running() hangs on AMD64 RHEL7 Refleaks 3.x)

2021-06-08 Thread Yury Selivanov


Yury Selivanov  added the comment:

>  MultiLoopChildWatcher must ensures that the event loop is awaken when it 
> receives a signal by using signal.setwakeup(). This is done by 
> _UnixSelectorEventLoop.add_signal_handler(). Maybe MultiLoopChildWatcher 
> could reuse this function, rather than calling directly signal.signal().

I think this is a good idea. MultiLoopChildWatcher could use setwakeupfd with 
some no-op callback just to wakeup the loop.

--

___
Python tracker 
<https://bugs.python.org/issue38323>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37146] opcode cache for LOAD_GLOBAL emits false alarm in memory leak hunting

2021-02-19 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Giving the ability to control the cache size, at least at Python startup, is 
> one option.

I'd really prefer not to allow users to control cache sizes. There's basically 
no point in that; the only practically useful thing is to enable or disable it.

--

___
Python tracker 
<https://bugs.python.org/issue37146>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

2021-02-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

> So you think that even a dedicated "LEN" opcode would not be any faster? 
> (This is getting in Paul Sokolovsky territory -- IIRC he has a variant of 
> Python that doesn't allow overriding builtins.)

Yeah, a dedicated LEN opcode could only be faster if it would not be possible 
to shadow builtins (or if there was a "len" operator in Python).  If that's not 
the case, this hypothetical LEN opcode would still have to check if "len" was 
shadowed or not, and that's slower than the optimized LOAD_GLOBAL we have now.

--

___
Python tracker 
<https://bugs.python.org/issue42990>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

2021-02-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I tried to implement such optimization in my old 
> https://faster-cpython.readthedocs.io/fat_python.html project. I implemented 
> guards to de-optimize the code if a builtin is overriden.

FWIW the globals opcode cache handles all of this now. There's no point in 
specifically optimizing the builtins lookup since we optimize all global 
lookups for a code object that's hot enough.

--

___
Python tracker 
<https://bugs.python.org/issue42990>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42927] Inline cache for slots

2021-01-14 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Some microbenchmarks:


Can you add a new one `read_slots`?

--

___
Python tracker 
<https://bugs.python.org/issue42927>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42927] Inline cache for slots

2021-01-14 Thread Yury Selivanov


Yury Selivanov  added the comment:

> So it seems that everything is in the noise range except the "float" 
> benchmark that is 1.11x faster

Yeah, this is why. 
https://github.com/python/pyperformance/blob/master/pyperformance/benchmarks/bm_float.py#L12

This is a great result, IMO. I'm +1 to merge this.

--

___
Python tracker 
<https://bugs.python.org/issue42927>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

2021-01-05 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Do we have good intuition or data about which operations need speeding up 
> most? Everybody always assumes it's BINARY_ADD, but much Python code isn't 
> actually numeric, and binary operations aren't all that common.

IMO, we shouldn't focus too much on optimizing binops. Regular webapps/network 
kind of code wouldn't benefit from that, and scientific code that uses numpy/ml 
libraries already offsets most of expensive computation to outside of CPython 
eval. Instead, I think, we should continue focusing on lowering the cost of 
function/method calls and attribute access.

--

___
Python tracker 
<https://bugs.python.org/issue42115>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

2021-01-05 Thread Yury Selivanov


Yury Selivanov  added the comment:

> The gist seems to be to have extra opcodes that only work for certain 
> situations (e.g. INT_BINARY_ADD). In a hot function we can rewrite opcodes 
> with their specialized counterpart. The new opcode contains a guard that 
> rewrites itself back if the guard fails (and then it stays unoptimized).

This is also roughly what I suggested in https://bugs.python.org/msg379333. 
Except that I don't think it's necessary to add new opcodes.

--

___
Python tracker 
<https://bugs.python.org/issue42115>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41891] asyncio.wait_for does not wait for task/future to be completed in all cases

2020-12-18 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 82dbfd5a04863d8b6363527e6a34a90c9aa5691b by Miss Islington (bot) 
in branch '3.9':
bpo-41891: ensure asyncio.wait_for waits for task completion (GH-22461) (#23840)
https://github.com/python/cpython/commit/82dbfd5a04863d8b6363527e6a34a90c9aa5691b


--

___
Python tracker 
<https://bugs.python.org/issue41891>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42600] Cancelling tasks waiting for asyncio.Conditions crashes w/ RuntimeError: Lock is not acquired.

2020-12-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

This has been actually first fixed in #41891 but somehow wasn't yet merged. 
Yurii, thanks so much for working on this and making a PR, there was just 
another PR to fix the same issue that was there first, so I had to merge that 
one.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
superseder:  -> asyncio.wait_for does not wait for task/future to be completed 
in all cases

___
Python tracker 
<https://bugs.python.org/issue42600>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41891] asyncio.wait_for does not wait for task/future to be completed in all cases

2020-12-18 Thread Yury Selivanov


Change by Yury Selivanov :


--
nosy:  -miss-islington
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue41891>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41891] asyncio.wait_for does not wait for task/future to be completed in all cases

2020-12-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

Thanks for the PR, Richard!

--
nosy:  -miss-islington

___
Python tracker 
<https://bugs.python.org/issue41891>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41891] asyncio.wait_for does not wait for task/future to be completed in all cases

2020-12-18 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 17ef4319a34f5a2f95e7823dfb5f5b8cff11882d by Richard Kojedzinszky 
in branch 'master':
bpo-41891: ensure asyncio.wait_for waits for task completion (#22461)
https://github.com/python/cpython/commit/17ef4319a34f5a2f95e7823dfb5f5b8cff11882d


--

___
Python tracker 
<https://bugs.python.org/issue41891>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42600] Cancelling tasks waiting for asyncio.Conditions crashes w/ RuntimeError: Lock is not acquired.

2020-12-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

Hey Hynek! :) Can you submit a PR?

--

___
Python tracker 
<https://bugs.python.org/issue42600>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

2020-12-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I will try to do some prototyping around that to see how much can we gain in 
> that route. In any case, adding LOAD_METHOD support for all kinds of calls 
> should be an improvement by itself even without caching, no?

Exactly.

As one argument for generalizing of LOAD_METHOD is that I, for example, try not 
to use kwargs in hot paths because I know it will be slower, which feels very 
wrong. I shouldn't care of internal implementation details of CPython and focus 
on writing readable code.

--

___
Python tracker 
<https://bugs.python.org/issue42115>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

2020-12-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I think I am closing the PR as it seems that the gains are not good enough 
> (and there is quite a lot of noise by comparing the benchmarks together).

IMO you need to implement LOAD_METHOD support for all kinds of calls, including 
the ones that use kwargs, to see any improvement.

--

___
Python tracker 
<https://bugs.python.org/issue42115>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42392] remove the 'loop' parameter from __init__ in all classes in asyncio.locks

2020-11-24 Thread Yury Selivanov


Yury Selivanov  added the comment:

Sorry, there are a few things in the committed patch that should be fixed. See 
the PR for my comments.

--
resolution: fixed -> 
stage: resolved -> patch review
status: closed -> open

___
Python tracker 
<https://bugs.python.org/issue42392>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42392] remove the 'loop' parameter from __init__ in all classes in asyncio.locks

2020-11-19 Thread Yury Selivanov


Yury Selivanov  added the comment:

> The safe code can look like:
>
> global_lock = threading.Lock()  like GIL

SGTM. We can use this strategy for all synchronization primitives and for 
objects like asyncio.Queue.

--

___
Python tracker 
<https://bugs.python.org/issue42392>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42392] remove the 'loop' parameter from __init__ in all classes in asyncio.locks

2020-11-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Despite the fact that asyncio.get_running_loop() never returns None but 
> raises RuntimeError (and maybe other tiny cleanups), 

It never raises when called from inside a coroutine. And I propose to do that. 
So it will never fail.

--

___
Python tracker 
<https://bugs.python.org/issue42392>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42085] Add dedicated slot for sending values

2020-11-18 Thread Yury Selivanov


Change by Yury Selivanov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue42085>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42085] Add dedicated slot for sending values

2020-11-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Is anything left to do?

I think we can close this now.

--

___
Python tracker 
<https://bugs.python.org/issue42085>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42392] remove the 'loop' parameter from __init__ in all classes in asyncio.locks

2020-11-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

Yeah, I follow the reasoning.

My use case:

   class Something:
  def __init__(self):
  self._lock = asyncio.Lock()

  async def do_something():
  async with self._lock:
  ...

And `Something` won't be created in a coroutine. So now I have to jump through 
the hoops to implement lazy lock instantiation.


> But the lock is tightly coupled with a loop instance. In other words, the 
> loop belongs to the loop.
>The lock cannot be used after the loop dies (stopped and closed).

I agree. Maybe the solution should be this then:


class asyncio.Lock:

   def _get_loop(self):
   loop = asyncio.get_running_loop()
   if self._loop is None:
self._loop = loop
   if loop is not self._loop: raise
   if not loop.is_running(): raise

   async def acquire(self):
   loop = self._get_loop()
   ...

This is what would guarantee all protections you want and would also allow to 
instantiate `asyncio.Lock` in class constructors, simplifying code.

--

___
Python tracker 
<https://bugs.python.org/issue42392>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42392] remove the 'loop' parameter from __init__ in all classes in asyncio.locks

2020-11-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

> That's good to know and I think more convenient to work with, so +1 from me. 
> I guess my remaining question though is whether it's okay to `await 
> lock.acquire()` on a single lock instance from multiple different running 
> event loops (presumably each in their own separate threads) or if there's a 
> need to restrict it to only one event loop.

No, it's not OK to use one lock across multiple loops at the same time. But 
most asyncio code out there doesn't have protections against that, and we never 
advertised that having multiple loops run in parallel is a good idea.  So while 
we could build protections against that, I'm not sure its needed. 

Andrew, thoughts?


> From what I've seen of asyncio user code, it seems reasonably common to 
> create async primitives (lock, semaphore, queue, etc.) in the __init__ for 
> some class prior to using the event loop, which would fail with usage of 
> `get_running_loop()` in the __init__ for the primitives. So, if it's not an 
> issue to wait until accessing the event loop until it's actually needed (e.g. 
> in the lock.acquire() or queue.get/put()), I think we should definitely try 
> to be conscious about when we call `get_running_loop()` going forward to 
> ensure we're not imposing arbitrary inconveniences on users.

Yep. This sums up how I think of this now.

--

___
Python tracker 
<https://bugs.python.org/issue42392>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42392] remove the 'loop' parameter from __init__ in all classes in asyncio.locks

2020-11-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

Oh my.  FWIW I think that we need to implement this differently. I don't think 
it matters where, say, an asyncio.Lock was instantiated. It can be created 
anywhere. So IMO its __init__ shouldn't try to capture the current loop -- 
there's no need for that. The loop can be and should be captured when the first 
`await lock.acquire()` is called.

I'm writing a piece of code right now that would need to jump through the hoops 
to simply create a new `asyncio.Lock()` in a place where there's no asyncio 
loop yet.

--

___
Python tracker 
<https://bugs.python.org/issue42392>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42392] remove the 'loop' parameter from __init__ in all classes in asyncio.locks

2020-11-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

Kyle, lmk if you want to work on this.

--
nosy: +aeros

___
Python tracker 
<https://bugs.python.org/issue42392>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42392] remove the 'loop' parameter from __init__ in all classes in asyncio.locks

2020-11-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

(or alternatively they can cache the running `loop`, but the first loop lookup 
should be performed with `asyncio.get_running_loop()`)

--

___
Python tracker 
<https://bugs.python.org/issue42392>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42392] remove the 'loop' parameter from __init__ in all classes in asyncio.locks

2020-11-17 Thread Yury Selivanov


New submission from Yury Selivanov :

asyncio.Lock and other primitives should no longer accept the `loop` parameter. 
They should also stop storing the current loop in the `self._loop` attribute. 
Instead, they should use `get_running_loop()` whenever they need to access the 
loop.

--
components: asyncio
messages: 381275
nosy: asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: remove the 'loop' parameter from __init__ in all classes in asyncio.locks
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue42392>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42140] asyncio.wait function creates futures set two times

2020-11-10 Thread Yury Selivanov


Yury Selivanov  added the comment:

Thanks for the change and for the discussion to everybody involved. The PR 
makes sense and I've already merged it.

--

___
Python tracker 
<https://bugs.python.org/issue42140>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42085] Add dedicated slot for sending values

2020-11-10 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 1e996c3a3b51e9c6f1f4cea8a6dbcf3bcb865060 by Vladimir Matveev in 
branch 'master':
bpo-42085: Introduce dedicated entry in PyAsyncMethods for sending values 
(#22780)
https://github.com/python/cpython/commit/1e996c3a3b51e9c6f1f4cea8a6dbcf3bcb865060


--

___
Python tracker 
<https://bugs.python.org/issue42085>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42085] Add dedicated slot for sending values

2020-11-10 Thread Yury Selivanov


Yury Selivanov  added the comment:

Vladimir, please do a follow up PR documenting Py_TPFLAGS_HAVE_AM_SEND.

--

___
Python tracker 
<https://bugs.python.org/issue42085>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24165] Free list for single-digits ints

2020-10-22 Thread Yury Selivanov


Yury Selivanov  added the comment:

Inada-san, how do you interpret the results? Looks like it's 
performance-neutral.

--

___
Python tracker 
<https://bugs.python.org/issue24165>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24165] Free list for single-digits ints

2020-10-22 Thread Yury Selivanov


Change by Yury Selivanov :


--
nosy: +pablogsal

___
Python tracker 
<https://bugs.python.org/issue24165>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42117] asyncio calls from sync/async, better docs or api support

2020-10-22 Thread Yury Selivanov


Yury Selivanov  added the comment:

I'm not sure what you're actually proposing, and only have a vague 
understanding of what you're complaining about. This is perhaps related to 
https://bugs.python.org/issue33523? If so then maybe leave a comment there?

--

___
Python tracker 
<https://bugs.python.org/issue42117>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

2020-10-22 Thread Yury Selivanov


Change by Yury Selivanov :


--
Removed message: https://bugs.python.org/msg379330

___
Python tracker 
<https://bugs.python.org/issue42115>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

2020-10-22 Thread Yury Selivanov


Yury Selivanov  added the comment:

> This idea can be implemented without opcode cache. I will try it.

I'd actually encourage trying to use the opcode cache because this way the 
optimization will be more generic. E.g. `decimal + decimal` would also be 
specialized via the cache, because you'd cache a pointer to the specific `+` 
operator implementation. I'm really not sure that adding specialized byte code 
is a good idea.

> * PHP uses scalar type for float and int

While at it, maybe we push the number of bits per int digit to 60?

--

___
Python tracker 
<https://bugs.python.org/issue42115>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

2020-10-22 Thread Yury Selivanov


Yury Selivanov  added the comment:

> This idea can be implemented without opcode cache. I will try it.

I'd actually encourage trying to use the opcode cache because this way the 
optimization will be more generic. E.g. `decimal + decimal` would also be 
specialized via the cache, because you'd cache a pointer to the specific `+` 
operator implementation. I'm really not sure that adding specialized byte code 
is a good idea.

> * PHP uses scalar type for float and int

While at it, maybe we push the number of bits per int digit to 60?

While at it, I'd also increase the digit size

--

___
Python tracker 
<https://bugs.python.org/issue42115>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

2020-10-21 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Imagine that we have a secondary copy of the bytecode in the cache inside the 
> code object and we mutate that instead. The key difference with the current 
> cache infrastructure is that we don't accumulate all the optimizations on the 
> same opcode, which can be very verbose. Instead, we change the generic opcode 
> to a more specialised to optimize and we change it back to deoptimize.

Yeah, I follow. As long as we keep the original list of opcodes we're good ;)

--

___
Python tracker 
<https://bugs.python.org/issue42115>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

2020-10-21 Thread Yury Selivanov


Yury Selivanov  added the comment:

Few thoughts in no particular order:

- I'd suggest implementing the cache for 2-3 more opcodes on top of the 
existing infrastructure to get more experience and then refactoring it to make 
it more generic.

- Generalizing LOAD_METHOD to work for methods with **kwargs, caching concrete 
operator implementations for opcodes like BINARY_ADD etc. are all possible on 
top of the current infra.

- Rewriting code objects in place is wrong, IMO: you always need to have a way 
to deoptimize the entire thing, so you need to keep the original one. It might 
be that you have well defined and static types for the first 1 invocations 
and something entirely different on 10001. So IMO we need a SpecializedCode 
object with the necessary bailout guards. But that's not a simple thing to 
implement, so unless someone will be working on this fulltime for a long time 
I'd suggest working off what we have now. (That said I'd take a close look at 
what Dino is building).

- There are multiple different approaches we can choose for optimizing CPython, 
ranging from hidden classes to a full blown JIT. I hope someone will do them 
one day. But IMO the current simple "opcode cache" (I wish we had a better 
name) mechanism we have would allow us to squeeze up to 15-25% median 
improvement in our benchmarks with relatively limited dev time. Maybe that's 
good enough for 3.10.

--

___
Python tracker 
<https://bugs.python.org/issue42115>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42113] Replace _asyncio.TaskWakeupMethWrapper with PyCFunction

2020-10-21 Thread Yury Selivanov


Change by Yury Selivanov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue42113>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-10-13 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset cfb0f57ff876ab3d04ff144f19eda58844981643 by Vladimir Matveev in 
branch 'master':
bpo-41756: Export PyGen_Send and wrap it in if-defs (#22677)
https://github.com/python/cpython/commit/cfb0f57ff876ab3d04ff144f19eda58844981643


--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-10-12 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Also, in Include/abstract.h it should only be available for limited C API >= 
> 3.10:

Vladimir, could you please submit a PR?

> We now should discuss the behavior of PySend_Iter() if value is NULL. It is 
> not documented, the current behavior differs from the behavior for non-NULL 
> value, and it is used in the ceval loop.

IMO: I'd keep the behavior and just document it.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-10-12 Thread Yury Selivanov


Change by Yury Selivanov :


--
status: closed -> open

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-10-12 Thread Yury Selivanov


Yury Selivanov  added the comment:

With the latest PR now merged this issue can be closed. Please reopen if there 
are any other action items left.

--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-10-09 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 037245c5ac46c3436f617a1f5d965929754be239 by Vladimir Matveev in 
branch 'master':
bpo-41756: Add PyIter_Send function (#22443)
https://github.com/python/cpython/commit/037245c5ac46c3436f617a1f5d965929754be239


--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-21 Thread Yury Selivanov


Yury Selivanov  added the comment:

> If you disagree with me, please say why, don't just merge the PR.

Apologies, Mark. I didn't intend to merge something bypassing your opinion; 
just missed your comment between reviewing multiple PRs in a few unrelated 
repos. I'm sorry.

On the actual naming subject, you proposed:

> `PySendResult PyIter_Send(PyObject *obj, PyObject *arg, PyObject **result);`

The problem with using this name is that ideally we should also support 
non-native coroutine and generator implementations (i.e. resolve the "send" 
attribute and call it using Python calling convention). Ideally we should have 
two C APIs: one low-level supporting only native objects and a high level one, 
supporting all kinds of them.

Can we perhaps add both `PyGen_Send()` and `PyCoro_Send()` for now that would 
only accept generators and coroutines respectively? After that we can discuss 
adding a more generic `PyIter_Send`?


> Would you revert the PR, please.

Since this is in 3.10/master that nobody uses right now except us (Python core 
devs), can we just issue a follow up PR to fix whatever is there to fix? I'd 
like to avoid the churn of reverting, and again, I apologize for pushing this a 
bit hastily.  Let me know if you actually want a revert and I'll do that.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-20 Thread Yury Selivanov


Yury Selivanov  added the comment:

Vladimir, could you please submit a PR to update 3.10/whatsnew? Need to mention 
both the new C API and the new perf boost in relevant sections.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

Thanks, Vladimir! Also huge thanks to Mark, Serhiy, and Stefan. If there are 
any nits to fix let's do that in follow up PRs.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-18 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 2b05361bf7cbbd76035206fd9befe87f37489f1e by Vladimir Matveev in 
branch 'master':
bpo-41756: Introduce PyGen_Send C API (GH-22196)
https://github.com/python/cpython/commit/2b05361bf7cbbd76035206fd9befe87f37489f1e


--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Sounds like a good middleground to start: add ``PySendResult `` and 
> `PySendResult PyGen_Send(PyGenObject*, PyObject* PyObject**)` specific to 
> generators and coroutines. 

Yes, it seems that everybody agreed on that. I can give the PR another review 
-- is it ready?

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-18 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I would also have preferred a more type specific function, but yeah, as long 
> as the types for which the function would normally be used are special cased 
> early enough in the implementation, it makes no big difference.

Maybe add two API funcs: PyGen_Send (specific to generators & coroutines) and 
PyIter_Send?

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Does it sound correct?

It does, but given the amount of back and forth on this, I'd wait for Serhiy 
and Stefan to confirm if they're OK.

IMO the `PyIter_Send` name is OK (given how generic the implementation will be) 
and returning an enum, while a tad unconventional for the C API is way more 
convenient for the developer.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-17 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I guess `PyIter_Send` would imply that this function should work for all 
> inputs (like in https://bugs.python.org/msg377007) which also sounds 
> reasonable.

I like `PyIter_Send` (and why I initially proposed it myself) because it will 
also work with the "send" slot defined on some types if we end up adding one.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-16 Thread Yury Selivanov


Yury Selivanov  added the comment:

> I think it should be specific to generators and coroutines. Calling 
> `PyObject_CallMethodIdOneArg(coro, _send, arg);` and interpreting 
> exceptions to emulate the low level API seems a bit too much.

To add to my point: typically higher-level APIs go under the `PyObject_*` 
namespace, whereas `Py{Type}_*` is more concrete. So I'd make `PyGen_Send` to 
only work with `PyGen` and `PyCoro`.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-16 Thread Yury Selivanov


Yury Selivanov  added the comment:

> Since coroutines inherit the generator protocol more or less, I think 
> "PyGen_Send()" is a more suitable name, better than "PyCoro_Send()".

OK, +1. PyGen_Send it is.

> Also should it be specific to generators/coroutines and accept PyGenObject*

I think it should be specific to generators and coroutines. Calling 
`PyObject_CallMethodIdOneArg(coro, _send, arg);` and interpreting 
exceptions to emulate the low level API seems a bit too much.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-16 Thread Yury Selivanov


Yury Selivanov  added the comment:

> As for returned value, I propose to return -1 in case of error, 1 for yielded 
> value and 0 for returned value (i.e. define PYGEN_RETURN = 0, PYGEN_YIELD = 1 
> and PYGEN_ERROR = -1, but without exposing public names).

Sure, that works.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-16 Thread Yury Selivanov


Yury Selivanov  added the comment:

Mark, Stefan,

I don't want this to be stale so I propose to move with my suggestions:

1. We make the new API public. Mark, if you have objections to that - please 
elaborate with some details. IMO, the corresponding Python API is long public 
and there's no harm in exposing a C version of it. Especially given the fact 
that uvloop, cython, and even asyncio itself will be relying on that API.

2. I propose to name the new API `PyIter_Send`. Motivation: it will work with 
both generators and coroutines and plays nicely with `PyIter_Next`.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-11 Thread Yury Selivanov


Yury Selivanov  added the comment:

@Mark
> Mark Shannon wrote: I don't think [the C-API function] should be public, as a 
> possible further improvement is to stop passing exceptions through a side 
> channel, but in result. Maybe we don't want to do that, but lets' not add to 
> the (already rather large) C-API.

Yeah, we can add it as a "private" function, I'm not entirely opposed to that. 
But... it would be great if Cython and C code could still depend on it and use 
it. And then... why should it be private? The corresponding Python API 
"gen.send()" and "gen.throw()" is public, why can't the C API be public too?

We will not fundamentally change generators (it would be a major backwards 
incompatible change), so committing to a good C API sounds reasonable.

@Mark
> Remember that PyIter_Next() is pretty much the same, though, and it has the 
> standard "return PyObject*" interface. These two would diverge then.

Maybe we should call it `_PyIter_Send()`?  While `.send()` is mostly about 
coroutines, regular generators have the method too, and it would be weird to 
call `_PyCoro_Send` on a generator object.

@Vladimir
> PYGEN_ERROR  | NULL | Regular PyErr_* functions should be used to work 
> with error case

Correct.

--

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41756] Do not always use exceptions to return result from coroutine

2020-09-10 Thread Yury Selivanov


Yury Selivanov  added the comment:

Big +1 from me. This is something I always wanted to do myself (since the time 
of PEP 492 & 525 implementations) and I think this is a necessary change. It's 
great that this isn't just a C API UX improvement but also yields a big perf 
improvement.

--
nosy: +Mark.Shannon, lukasz.langa, vstinner

___
Python tracker 
<https://bugs.python.org/issue41756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41733] ContextVar get value is unexpected

2020-09-08 Thread Yury Selivanov


Yury Selivanov  added the comment:

> my expected result is all objects should be different, because the code runs 
> in different context.

Contexts "branch", so if you store something in the context before asyncio.run 
it will store it in the current thread state. And then asyncio.run will pick it 
up.

Also I'd recommend using something else than object IDs - those can be reused 
and can be very confusing when browsing logs -- it might look like it was the 
same object, but in fact it could be to different objects sharing the same ID 
at different points of time.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed
type:  -> behavior

___
Python tracker 
<https://bugs.python.org/issue41733>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41687] sendfile implementation is not compatible with Solaris

2020-09-05 Thread Yury Selivanov

Yury Selivanov  added the comment:


New changeset 8c0be6fd9101746235b63ddfb84106d1e9ca286b by Jakub KulĂ­k in branch 
'master':
bpo-41687: Fix sendfile implementation to work with Solaris (#22040)
https://github.com/python/cpython/commit/8c0be6fd9101746235b63ddfb84106d1e9ca286b


--
nosy: +yselivanov

___
Python tracker 
<https://bugs.python.org/issue41687>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41687] sendfile implementation is not compatible with Solaris

2020-09-05 Thread Yury Selivanov


Change by Yury Selivanov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
type:  -> behavior

___
Python tracker 
<https://bugs.python.org/issue41687>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39010] ProactorEventLoop raises unhandled ConnectionResetError

2020-09-03 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 40e2444c364eede59f193979df5a02ed958f56e9 by Miss Islington (bot) 
in branch '3.8':
bpo-39010: Improve test shutdown (GH-22066) (#22083)
https://github.com/python/cpython/commit/40e2444c364eede59f193979df5a02ed958f56e9


--

___
Python tracker 
<https://bugs.python.org/issue39010>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41696] asyncio.run interacts surprisingly with debug mode

2020-09-03 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset a2680058ee1f43cab282e9f4c2b35fd89464ebb6 by Miss Islington (bot) 
in branch '3.9':
bpo-41696: Fix handling of debug mode in asyncio.run (GH-22069) (#22071)
https://github.com/python/cpython/commit/a2680058ee1f43cab282e9f4c2b35fd89464ebb6


--

___
Python tracker 
<https://bugs.python.org/issue41696>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41696] asyncio.run interacts surprisingly with debug mode

2020-09-03 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 1f5f12737755f246fee83a54c600779ad76934a4 by Miss Islington (bot) 
in branch '3.8':
bpo-41696: Fix handling of debug mode in asyncio.run (GH-22069) (#22072)
https://github.com/python/cpython/commit/1f5f12737755f246fee83a54c600779ad76934a4


--

___
Python tracker 
<https://bugs.python.org/issue41696>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39010] ProactorEventLoop raises unhandled ConnectionResetError

2020-09-03 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 8d68e59f11267ded765d4af6d71a49784a12fad5 by Miss Islington (bot) 
in branch '3.9':
bpo-39010: Improve test shutdown (GH-22066) (#22082)
https://github.com/python/cpython/commit/8d68e59f11267ded765d4af6d71a49784a12fad5


--

___
Python tracker 
<https://bugs.python.org/issue39010>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39010] ProactorEventLoop raises unhandled ConnectionResetError

2020-09-03 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset a986b061a3cd4661eb9f857e2936291f1847bd15 by Miss Islington (bot) 
in branch '3.8':
bpo-39010: Fix errors logged on proactor loop restart (GH-22017) (#22035)
https://github.com/python/cpython/commit/a986b061a3cd4661eb9f857e2936291f1847bd15


--

___
Python tracker 
<https://bugs.python.org/issue39010>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39010] ProactorEventLoop raises unhandled ConnectionResetError

2020-09-03 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 49571c0b0e57e20d85727c738d9a0fe342dd2938 by Miss Islington (bot) 
in branch '3.9':
bpo-39010: Fix errors logged on proactor loop restart (GH-22017) (#22034)
https://github.com/python/cpython/commit/49571c0b0e57e20d85727c738d9a0fe342dd2938


--

___
Python tracker 
<https://bugs.python.org/issue39010>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39010] ProactorEventLoop raises unhandled ConnectionResetError

2020-09-02 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset be435ae2b064dc64f04475bec632862e1dbf605f by Ben Darnell in branch 
'master':
bpo-39010: Improve test shutdown (#22066)
https://github.com/python/cpython/commit/be435ae2b064dc64f04475bec632862e1dbf605f


--

___
Python tracker 
<https://bugs.python.org/issue39010>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41696] asyncio.run interacts surprisingly with debug mode

2020-09-02 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset 0770ad948cb6d9f7f6c4002efd83e27c27069808 by Shantanu in branch 
'master':
bpo-41696: Fix handling of debug mode in asyncio.run (#22069)
https://github.com/python/cpython/commit/0770ad948cb6d9f7f6c4002efd83e27c27069808


--

___
Python tracker 
<https://bugs.python.org/issue41696>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41696] asyncio.run interacts surprisingly with debug mode

2020-09-02 Thread Yury Selivanov


Yury Selivanov  added the comment:

Yes, this is a bug. The default for the `debug` parameter should be `None` and 
it should only call `loop.set_debug` if a non-None bool value was explicitly 
passed. Care to submit a PR?

--

___
Python tracker 
<https://bugs.python.org/issue41696>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39010] ProactorEventLoop raises unhandled ConnectionResetError

2020-08-31 Thread Yury Selivanov


Yury Selivanov  added the comment:


New changeset ea5a6363c3f8cc90b7c0cc573922b10f296073b6 by Ben Darnell in branch 
'master':
bpo-39010: Fix errors logged on proactor loop restart (#22017)
https://github.com/python/cpython/commit/ea5a6363c3f8cc90b7c0cc573922b10f296073b6


--

___
Python tracker 
<https://bugs.python.org/issue39010>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



  1   2   3   4   5   6   7   8   9   10   >